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This report presents the results of a field test, the 
purpose of which vas to determine the effects on school system 
management, decisions, and operations of the introduction of 
specially prepared information on the relative achievement levels of 
schools and grades serving students of similar economic levels. A 
relatively simple and economical system of visually unmistakable 
signals was introduced so that decisions, especially at managerial 
levels in a large urban school system, vould not have to be made in a 
vacuum — that is, vithout clear indications of schools and grades 
nhere unusually high or lev performances vere being achieved. The 
introduction of the signals during the 1972-73 school year, vith 
excellent cooperation by Atlanta school personnel, found in part that 
school officials readily accepted the signals as an understandable 
mechanism that they could use to diagnose performance; personnel 
division staff used signals to improve procedures for screening nev 
applicants; but that signals seemed to play little or no part in 
decisions regarding instructional programs and textbook adoption. One 
major implication of the findings for large urban school systems is 
that decisions concerning the operation of the school vhich are made 
higher up in the administrative structure are generally made without 
information regarding performance at the classroom level. 
(Author/DN) 
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IMPROVING THE MANAGEMENT OF PUBLIC SCHOOL SYSTEMS: 
A TEST OF SIGNALS OF RELATIVE SCHOOL PERFORMANCE 

A. INTRODUCTION 

This report presents the results of a field test conducted by the 
Program Evaluation staff of The Urban Institute In the Atlanta Public School 
System during the 1972-73 school year. The purpose of the test was to deter- 
mine the effects on school system management, decisions, and operations of 
the introduction of specially prepared information on the relative achieve- 
ment levels of schools and grades serving students of similar economic levels. 

Student performance has become an issue in many large urban school 
systems. In Atlanta, for example, the school system is now required to make 
public school by school achievement test results. School systems have tried 
a variety of approaches in attempts to inq)rove student performance, but 
school administrators typically do not know how successful their own efforts 
or the efforts of others have been. The federal government has required 
evaluation of Individual projects and In so doing has added to local interest 
in student performance, but information frorn^ federally required evaluations 
has generally played little or no part in school system planning or admin- 
istration. 

Officials with major management responsibilities In large school 
systems often do not know precisely where to look for success or failure. 
Present Indicators of success or failure in the classroom grow progressively 
weaker as distance from the classroom Increases, Part of the problem stems 
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from the absence both of clearly stated goals and objectives for education 
and of agreed-upon measures of success In meeting those goals. Even In the 
limited area of a school system's success In teaching basic skills, however, 
the parties to a discussion of success or failure rarely use the same set of 
assumptions and measurements In Identifying success or failure, and the data 
•used In any discussion are rarely current. 

In a large school system, the duties of management above the level of 
the school principal Incliide many support functions not directly tied to 
the performance of the students. Support functions such as payroll, facll-- 
Itles, equipment, personnel functions, ensuring compliance with state and 
federal lavs, and purchasing offer economies of scale In a large system and 
are generally performed by centralized units. 

Our examinations of large school systems have found organized manage- 
ment In the support area, but no well-defined management structure aimed at 
systematically Improving student performance. The setting of minimum 
acceptable levels of performance and assurance of quality In the areas of 
Instruction and curriculum are generally accepted as system-wide functions. 
Management tools for applying even these limited standards, however, are 
rarely available. The reason generally given for this lack ±9 the diffi- 
culty of creating performance Indicators that are current, readily Inter- 
pretable, accessible to decision makers, and yet specific enough to be the 
basis for decisions and actions aimed at improved performance. 

While The Urban Institute continued to explore the question of what 
would constitute school system management approaches aimed at improving 
performance, an early effort was made to remedy the lack of accessible and 
Interpretable performance information. This was done in order to test the 
extent to which lack of information about student performance has impeded 
^ efforts to improve performance in large school systems, 
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Our exploratory effort Into these management questions was initiated 
in January 1971, using Urban Institute research funds, and expanded in 
January 1972, when support from the Office of Economic Opportunity was 
obtained. Phase I of that effort (ending in October 1972) produced usable 
performance information.^ The technique used existing school system data 
on student achievement and participation in the subsidized school lunch 
program to locate specific grades in individual elementary schools in which 
the level of average student performance was significantly better or worse 
than the level of performance in schools of similar economic level. 

The Phase II work reported here (November 1972 to November 1973) was 
designed to make information on relative school performance widely available 
In a school system and to observe the influence of this information on 
decision making in areas related to student performance. Signals of rela- 
tive school performance covering school years 1970*71 and 1971-72 were 
created and introduced into the school system in the fall and early winter 
of 1972, and a detailed study was made of the impact of that information on 
the actions of school officials. 

Two results were expected from this effort. First, a test would be 
made to determine if the provision of performance information to an operating 
school system produces different management decisions. Second, in the 
process of that test, better information would be acquired on how the school 
system management actually attempts to influence performance. 

Since January 1971, staff from The Urban Institute and the Atlanta 
Public School System have worked together to define and shape various 
aspects of this project, including the work reported here. Although Infor- 
mation from other school systems and other research results have also been 

1^ See Bay la F. White et al . , The Atlanta Project; Develdping 
Q^ i^als of Relative School Performance , The Urban Institute, October 1972. 
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of value » It vas decided early in our research efforts to use an actual large 
school system as a "laboratory" In order to force the development effort to 
deal with reality. The Urban Institute staff chose Atlanta from among several 
large urban school systems because It tested all students twice yearly, had 
sufflclen.: data available to explore a variety of performance questions » 
maintained a strong aystemwide Interest in improving performance^ and had 
personnel who would fully support the basic aims of the project. This co- 
operation has never wavered and, as a result of this relationship, we are 
now in a position to discuss large urban school systems as they acttially 
function. 

B. THE URBAN INSTITUTE APPROACH 

School administrators are overwhelmed with numbers. All too often, the 
response of researchers has been to create new and different numbers. The 
approach used here was to make comparisons of the relative performance of 
schools serving similar students, identify significant cases of extreme per«^ 
formance, and display the results as a series of charts in which ted "signals" 
denote levels of relatively low performance and blue "signals" denote levels 
of relatively high performance.^ This signaling technique permits compact 
displays of information on the relative performance of every grade in a school 
for several years and for different subject areas or for every grade in every 
school in an administrative area for successive years. In this way, a large 
amount of information of high Interest to the school system management can be 
extracted from the existing data and provided In a compact, accessible form* 

^2^ See Chapter II, p. 25 ff . for a description of the technique used 
to derive and display "signals." 
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Even thougl^ there Is much argument about the applicability and accuracy 
of achievement tests, there is general agreement that students in elementary 
schools should learn to read and to solve simple arithmetic problems. By 
using relative comparisons and by being conservative in setting boundaries 
for what is to be considered "extreme" performance, this approach provides an 
answer to a question of great Interest to school system administrators: "Where 
exactly are our successes and our failures?" 

The following hypothesis was then tested by the Urban Institute in Atlanta 

during the past year; 

Information on relative performance, when introduced into 
a sufficiently well-understood planning and management structure, 
does (or does not) produce measurable changes (a) in the decision- 
making process of a local school system, (b) in the decisions made 
by school officials, and (c) ultimately, in school performance. 

In Its simplest form, the hypothesis should be viewed as a test of whether 
the provislou of performance information alone is sufficient to produce pur- 
poseful systemwide changes in management in the area of performance. Such 
information is certainly a necessary condition, f( r without usable performance 
information, decisions cannot be based on the results of past actions. The 
presence of good performance information alone, however, may not be sufficient 
to cause a large school system to change its decision-making processes. To 
test the effects of relative school performance information on a large school 
system, compact performance information was made available, and a detailed 
study was made of the Impact of the information upon system operations. 

This year of study focused on those portions of the school system which 
have some Immediate, direct Impact on the classroom and might therefore respond 
to the new Information within the short time span of this study. Based on the 
earlier work in Atlanta, it had become apparent that the Introduction of signal 



erJc 



6 



information would not have an equal impact on the operation of all parts of 
the school system. Administrative units were identified for study on the 
basis of two criteria. First, there had to be some direct interaction between 
the unit and the classroom. Second, the administrative unit had to have some 
flexibility and discretion to react to signal information. Moreover, Urban 
Institute staff and our advisory board in Atlanta selected for study the 
management of the following three activities: 

• Recruitment, assignment, and reassignment of teaching staff 

• Design of the instructional program «md provision of instructional 
materials 

• Improving the skills of classroom teachers. 

The study attempted to learn how these activities are normally conducted in 
Atlanta, what impact the introduction of signals had on these activities, and 
the resulting effect upon performance in classrooms. Research into these 
activities was based on extensive structured interviewing, surveys, formal 
reporting by Atlanta Public School personnel, and attendance by Urban Institute 
staff at numerous operational and staff meetings in Atlanta. 

In considering the information presented below, it is Important for the 
reader to keep in mind two facts about the activities presently occurring 
within the school system: 

• Actions taken to Improve performance must somehow do so 
through their effect on the education taking place in the 
classroom. In the material below, each management activity 
is examined in terms of the impact of the activity on the 
classroom and in terms of the use or non-use of informa- 
tion about relative school performance. 

• There is no agreement on the extent to i^ich these activities 
should rely on performance information. No one yet knows 
precisely how performance information can best be used to 
manage school activities or even which management approaches 
will result in Improved student performance. 
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C. SCHOOL SYSTEM MANAGEMENT AND PRACTICES 
BEFORE THE INTRODUCTION OF SIGNALS 

This section summarizes the operational characteristics of the school 
system before the introduction of signals. A goal of The Urban Institute 
research effort has been to understand and Improve the management of large 
urban school systems. It is Important, in order to do this, to understand 
how school systems are managed at present. Not only has the Atlanta School 
System provided an excellent source of data for developing the signals, it 
has also provided The Urban Institute with a unique opportunity to observe 
closely the operations of a school system and to examine how management 
decisions are made and Implemented. Atlanta Public Schools staff members 
have fully cooperated in providing Information on their activities, greatly 
facilitating our collection of operational data. Independent of our findings 
on the effects of signals on the operation of the school system, several 
factors have been identified concerning the management of school systems 
which appear to have significance for Atlanta and other large urban school 
systems. 

Atlanta, like most large school systems, operates through a line manage- 
ment chain that starts with the classroom teacher. The chain goes up through 
school principals to an Area Superintendent, and on to the Superintendent and 
his Immediate staff, who report to an elected School Board. Specialized 
support functions such as personnel, finance. Instructional services, re- 
search and development, and buildings and facilities are handled by separate 
staff divisions headed by Assistant Superintendents who report directly 
to the Superintendent. Area Superintendents are responsible for the opera- 
tion of the school system within their geographic Areas. In Atlanta, the Area 

Office has a support staff composed primarily of Resource Teachers. These 

O 
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Resource Teachers can be characterized as experienced teachers vho provide 
specialized assistance to the principals and teachers In Individual schools. 
The school system Is managed through a direct line organization, supported 
by a fairly large and complex bureaucracy with distinct organizational lines 
and some overlapping functions. 

General observations made on school system operation In the period 
before signals were Introduced Include the following: 

• Information on the performance of students was, In general, 
neither available to nor used by management units o'iiher than 
the classroom teacher, principals and Resource Teachers. ^ 
Available performance data x'ealt for the most part with 
absolute performance levels, rather than with Information 

on variations In performance among schools aud grades serving 
similar chlldrerr. 

• In general, mechanisms were lacking for activities of the 
central school system management to be tailored to actual 
variations In performance among or within schools. 

There have been attempts by various organizational units or programs to 
apply resources selectively to particular grades and schools; however. In 
nearly every case examined, either the thrust of the activity was diminished 
or Its direction was altered as the activity filtered down to the classroom 
level . 

Several more specific results are cited below to support these general 
conclusions about the operations of the school system prior to the Intro- 
duction of signals. While these specific results may seem unique to Atlanta, 
evidence frciu other cities Indicates that the general results are applicable 
to other large urban school systems as well: 



3. An exception to this was the Comprehensive Instructional Program 
(GIF) which had created a system for supplying teachers directly with In- 
formation about their students. GIF has also used achievement data In 
attempts to evaluate Its own efforts. 

Atlanta's Research and Development Department has generated and 
distributed absolute performance Information and relative gains during the 
course of this study. This information, which is complementary to signals, 
^ IS also distributed in 1972-73. 
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• In the personnel area: 

— New teachers have been hired and initially assigned without 
any attempts to use the specific performance record of the 
group of students eventually taught. 

— Principals have no advance information about the character- 
istics of teachers new to the school (e.g«, information on 
the teachers* instructional strengths and weaknesses). 

— Principals have little Input into decisions about "^hicli 
teachers will be assigned to their schools; those making 
the decision do not have available student performance 
information in a form that could be useful in making the 
decision. 

— The major factor which has governed staffing assignments 
is the need to meet the requirements of the court-ordered 
racial ratios in individual schools. Given this constraint, 
neither performance information nor the mechanisms to use 
such information was evident in decisions about the staff- 
ing of schools. 

• In the instructional area: 

— The Instruction Division prepares the same curriculum 
guides for use in all schools. Once the guides have 
been circulated among schools, little or no Information 
on the implementation of the curriculum reaches the 
Instruction Division, No formal mechanism exists to 
assess good practices among schools or faults in the 
curriculum • 

— The best example in Atlanta of the use of performance 
information occurs in the Comprehensive Instructional 
Program, which has developed a test for diagnosing the 
progress of students in mastering basic skills. Results 
of these tests, in the form of Individual student pro- 
files, have been provided directly to the classroom 
teacher. While the instructional process in the class- 
room could benefit directly from the teacher's use of 
these data, no comparable stream of performance data 
existed for use by the central staff in the design of 
instructional components. 

— Each Area Office has been assigned a staff of Resource 
Teachers, who are curriculum specialists and who spend 
a substantial portion of their time working with class- 
room teachers to Improve the level of instruction. No 
assessment has been made of the effect those Resource 
Teachers have had on student performance. 
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— A textbook committee approves texts for use in the 

classroom, and the Textbook Coordinator on the Instruction 
Division staff advises individual principals on choosing 
textbooks. Both the textbook committee and the Textbook 
Coordinator operate without empirical information on variations 
In performance of Atlanta students using the various texts. 

— Directors of special programs (e.g.. Title I) design 
programs and make changes in programs generally using 
little information about the effects on the performance 
of students in the schools in which the programs are 
Implemented » even though some information of this type 
is generated. 

In the staff Improvement area: 

— Although Resource Teachers make decisions on which 
grades (and sometimes which schools) to visit, these 
decisions are seldom based on relative student achieve- 
ment information. 

— No central records have been kept of the university courses, 
Instruction Division courses or Area in-service programs 
that teachers attend. The Personnel Division keeps some 
records for salary purposes. The effect of such training 
on student performance is unknown. 

— Neither principals nor Area Superintendents are systemat- 
ically informed of the teachers who complete courses. 
This has made Intuitive evaluations by school system 
personnel of the effects of such courses Impossible. 

— Proficiency nodules, a newly Introduced teacher training 
program, are available to all teachers. Although suc- 
cessful completion of the module program entitles a 
teacher to a salary increment, no effort has been made 
to determine what effect completion of the modules has 
on student performance. 

— A rating instrument designed to appraise classroom teachers' 
proficiency in the teaching of reading has recently been 
Introduced. This instrument has not been validated to 
determine if its ratings correlate with the teacher's 
ability to improve the performance of the students. 

— Classroom teachers and Resource Teachers seem to have 
little or no input into decisions made by the Instruction 
Division about in-service training offered. 

—Neither the Instruction Division nor the Area Offices use 
student performance as a measure of the effectiveness of 
the courses or programs they offer to teachers. 
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These findings summarize the situation with respect to the avail* 
ability and use of relative performance information. The findings do not 
represent an evaltiation of the management activities of the Atlanta School 
System or any other system, nor has the feasibility of evaluating manage- 
ment activities through the use of relative student performance in the 
schools been established. What this preliminary work does represent is 
a survey to determine who might be interested in relative performance 
information as well as an attempt to establish how management efforts are 
presently organized. Our information to date indicates that management 
activities as described here are conducted in a similar manner in many 
school systems. Given this management environment, one could expect 
that personnel in a school system might respond favorably to the concept 
of signals displaying performance information— as th&y did. Most are 
committed to Improving performance. However, one can also begin to 
anticipate that the chance of an effective management response from a 
school system's staff and line management elements to performance inform- 
ation might be low — as, in fact, it turned out to be. These results are 
discussed in the next section. 

D. THE EFFECT OF INTRODUCING SIGNALS OF RELATIVE PERFORMANCE 
ON THE OPERATION OF A LARGE URBAN SCHOOL SYSTEM 

The major thrust of the past year's study was to make relative per- 
formance information available to managers and administrators of the school 
system and to determine what effect the availability of the information had 
on the operation of the school system. Atlanta school officials cooperated 
wholeheartedly in the distribution and explanation of the signals and in 
facilitating our collection of information to document the effects observed. 
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Four general results which characterize the findings of this part of the 
study are the following: 

• School officials readily accepted the signals and often had 
words of high praise for the signals as an understandable 
mechanism which they could use to diagnose performance in 
the school system and plan program activities. 

m In only a few cases did these same officials give evidence 
of the actual use of relative performance information in 
making significant management decisions. 

• There was little evidence of a shift of specialized school 
system activities to grade levels In schools which were 
signaled as extremely high or low in relative performance. 

• In the few cases where activity did seem to be dispropor- 
tionately allocated to grades having a particular type of 
signal, such as in a particular Area or with a particular 
Resource Teacher, there were converse allocations of effort 
which demonstrated that the signals were not having a con- 
sistent effect. 

These general results are drawn from specific obser>rations that were made 
throughout the school system on how and what decisions were made, how and 
to what extent they were Implemented and what, if any, effect they had on 
classroom activities. Some more specific results of our analyses in the 
three areas on which we concentrated our attention are listed below: 



• Personnel : 

— The Personnel Division staff showed an interest in using 
signals to Improve procedures for screening new applicants 
and for reassigning current staff. There was, however, no 
evidence that anyone followed through on this interest and 
developed new screening or assignment techniques based on 
the signals. 

— Since no new elementary teachers were hired between January 
and September 1973, signals could not play a role in that 
activity this year. 

—Area Superintendents, who have authority over all transfers 
In teaching assignments in their respective Areas, generally 
did not use signals in making these decisions. 
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— One Area Superintendent did reassign several teachers who had 
been in a school with several red signals to schools with 
several blue signals In the hope that the teachers would 
be Influenced by the high performance of teachers In their 
new schools. These assignments became effective In 
September 1973. 

Instructional Program: 

— In general, no evidence was detected of cases where signals 
were considered In decisions Involving the shape of the 
Instructional program for the school system. 

— ^More particularly, neither performance Information In general, 
nor the signals In particular, seem to have affected the choice 
of pilot schools for the new elementary school curriculum or 
the appraisal of the program as It Is being Implemented. 

— The Instruction Division staff responsible for the Com- 
prehensive Instructional Program (CIP), a program started 
three years ago to provide Intensive assistance to teachers 
and students In reading and arithmetic, were familiar 
with signals but gave no Indication of having used signals 
In the operation of the program. (In the past, CIP 
activity was evaluated with achievement test data, but 
this has been discontinued.) 

— ^Although several principals reported making decisions on the 
adoption of new textbooks this year for the coming years, 
signals played no part In the decisions about particular texts 
or materials. 

— In general, neither Area Superintendents, Curriculum Coor- 
dinators nor principals reported using signals or other 
performance Information In discussions about the Instructional 
programs — either In general or for specific schools or grades. 
In a few cases there was evidence of a principal using signals 
to support decisions on restructuring a particular Instructional 
program. 

Improving the Skills of the Teaching Staff: 

— Two mechanisms for Improving teacher skills were examined 
In depth: In-servlce training and direct assistance to 
classroom teachers through the Area Office Resource Teachers. 
In neither case Is there evidence to support a conclusion 
that signals had a general, systematic and consistent effect 
on decisions concerning these activities or on the pattern of 
activities that resulted. 
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— In particular, except for one or two principals, no Atlanta 
school official reported using Information from the signals 
In designing In-servlce programs or In determining who should 
attend them. 

—When the school system Is considered as a whole, the 

distribution of Resource Teacher activity to grades receiving 
signals Indicating high, low or average relative performance 
Is nearly Identical to the proportion of grades having the 
various types of signals. That Is, the observed distribution 
of assistance showed no tendency to disproportionately favor 
grades signaled as having a record of extreme relative per- 
formance — either high or low. 

— When the distributions of Resource Teacher assistance ar& 
considered on an Area Office basis, statistically significant 
cases of disproportionate assistance being given to grades 
having a particular type of signal are detected. Only in 
Area III, however, were the differences large enough to have 
any operational significance. 

— In those grades visited by Resource Teachers, grades having 
a signal associated with low relative performance did tend 
to receive a bit more assistance in terms of time spent 
providing the assistance and of the number of occasions on 
which assistance was provided, but the differences appear 
to be due to a few Resource Teachers and a few grades, and 
the overall differences are not statistically or operationally 
significant. 

— Thus, while there are instances where Resource Teachers 
were users of signals and disproportionately provided 
assistance to grades having a particular type of signal, 
there is no evidence of a systematic or pervasive tailoring 
of the delivery of Resource Teacher assistance in response 
to the signals of relative performance. 



The results of our analyses are discussed in greater detail in 
Chapter V of this report. The implications of these findings, for 
Atlanta and for other large school systems are addressed in the next 
section. 
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E. IMPLICATIONS OF WHAT WE HAVE LEARNED 



Our work In and with the Atlanta School System has several implications 
for the operation of large urban school systems: 

• Decisions concerning the operation of the school which are made 
higher up in the administrative structure are generally made 
without Information regarding performance at the classroom level 
and the variation that exists at this level, whether this infor- 
mation Is available or not. Furthermore, there do not appear 

to be well defined management mechanisms or arrangements 
which can be utilized by staff or line units in responding 
to relative performance information. 

• In the past, relative performance information was not 
available in an immediately usable or understandable form 
for use by school system decision makers. [Some use had 
been made of absolute performance data.] 

• The Urban Institute has provided relative performance infor- 
mation which school off icials. accepted and praised; however, 
this information was not generally used in making operational 
decisions nor did it have a consistent or obvious effect cn 
the activities of the aystem. Further, management mechanisms 
for making use of performance information in decisions 
intended to affect performance in the schools either do not 
exist or are at cross purposes with each other. 

• Further research is necessary to say in which parts of a 
school system the use of performance information is actually 
feasible and to design the management techniques for bringing 
such information to bear on decisions in a systematic way. 

Several other Implications can be drawn: 

• Since signals were praised and accepted by most school 
officials and actually utilized in a few instances, the 
continued publication of signals on an in-house basis may, 
through time, result in the movement towards a management 
system based on performance information. This is a viable 
possibility since the costs of production are relatively low. 

0 At a minimum, however, it would seem necessary at least to 
develop a unified strategy for various components of the 
system to use either the signals or other relative perfor- 
mance Information. 

In order to have some degree of confidence In achieving 
better performance management capabilities for large urban 
school systems, it appears necessary to adopt a two-pronged 
approach. First, improve the form and accuracy of Information 
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which can be created from measures of student performance. 
Second » develop the management techniques and approaches 
which will enable school officials to use such information 
in their efforts to improve student performance throughout 
the school system. 

F. SIGNALS AMD SIGNALING 

As has been described above (and in more detail in Chapter II) » "signals" 
were used:'as a mechanism to display the relative performance of individual 
grades in each elementary school. There are five levels of relative per-* 
formance which enable a massive amount of achievement data to be displayed 
in a compact » understandable manner % 

The past year's work has demonstrated that school systems could produce 
signals in-house without large expenditures of resources^ if achievement 
test data and indication of the socioeconomic level of the schools or grades 
are available in machine-processable form. Documentation for computer 
programs that have been used to produce the signals and statistical analysis 
of the data is available as an appendix to this report. 

During the last year^ Atlanta has modified its computer processing 
system and now can produce grade level averages for each school. A small 
change in their subsidized school lunch computer program would allow the 
calculation of the percent of students receiving subsidized school lunches on 
the basis that has been used in this system. With this information already 
available 9 only the signal preparation programs need be run. Our present 
program typically can produce signals from this Information with one day of 
analyst time and perhaps $100-$200 of computer time. 
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For a school system similar In size to Atlanta, which has a data base 
of Individual student raw scores and some measure of socioeconomic status » 
the production of signals vould take two to four weeks following the grading 
of the tests and would cost on the order of ^ $3,000 (excluding printing 
costs). First year costs may be slightly higher, but on-going operational 
costs should be no more than this amount. 

The analyses conducted on the signals this year have also demonstrated 

that: 

• The technique used to Identify schools performing high or 
low relative to schools having students from similar socio- 
economic conditions can indicate extreaies of performance 
across the full range of socioeconomic conditions existing 
in the Atlanta student population. 

• Gr;?.des that are signaled actually do represent extremes of 
performance. Individual test scores from grades that were 
signaled were examined in detail to verify that the variation 
observed for the grade was not just a random event that could 
statistically be expected to occur in the number of cases 
detected. In fact, signaled differences are generally statis- 
tically significant and cannot be attributed to chance 

. variation. 

• Within their error range, the signals appear to be able to 
identify a recurring phenomenon. The probability of a grade 
having a certain signal in one year is dependent on the sig- 
nal it received during the previous year. These conditional 
probabilities appear to exhibit some stability through time. 
There were few significant differences in the transition 
probabilities between academic years 1970-71 and 1971-72 and 
between the years 1971-72 and 1972-73. If a school system 
can detect and react to such phenomena, then it has the 
potential for taking actions aimed at affecting these tran- 
sitions and achieving higher student performance. 

While the analyses during this phase of the study have confirmed the 

technical soundness and utility of the signals, they have not attempted to 

exploit fully the information about relative performance that can be extracted 

from data collected by the school system. Further work is also needed to 

establish the feasibility of managing to Improve performance. Such analyses 

rere not the principal foci of this phase of the study; the majority of our 



i 18 

effort was expended on analyzing the existing i&anagement of a large school 
system and the effect of the Introduction of performance Information in the 
form of signals. Questions of what types of organizational arrangements are 
both feasible and desirable and to what extent each could be supported with 
accurate information about relative performance are the logical next steps 
of this effort. Sufficient information has already been collected to support 
most of this work. 

G. ORGANIZATION OF THE REPORT 

The remainder of this report is divided Into four chapters. The 
reasons underlying the development of signals of relative performances the 
techniques used in that development, and properties of the signals are 
described in Chapter II. Chapter III explains the general setting in which 
the study took place and the procedures used in the study. In Chapter IV ^ 
the operation of the Atlanta School System is discussed and an examination 
of the management of activities chosen for investigation is presented. 
Chapter V presents the results of those investigations. 
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CHAPTER II 
SIGNALS OF RELATIVE PERFORMANCE 

While the remaining chapters of this report describe the Impact of new 
information on the management of a large school system, this chapter focuses 
on how and why that new Information was developed. The chapter begins with 
a discussion of the philosophy underlying the focus on relative performance. 
The technique for generating signals of relative performance is explained. 
Some of the properties of signals are analyzed In the final section. 

A. ASSUMPTIONS UNDERLYING THE DEVELOPMENT 
OF SIGNALS OF RELATIVE PERFORMANCE^ 

The ultimate aim of this project Is to improve the capacity of a large 
school system to plan and manage its activities in a manner which will make 
better use of available, limited resources. The technique proposed provides 
school officials with information about the relative performance of students 
In schools and grades within those schools throughout the school system. 
The information on relative school or grade performance is derived from 
existing data assembled in concise, simple displays which signal how well or 
badly a school is teaching basic skills. 

Why produce signals at all? Of what value are such signals? The de- 
velopment of signals of relative performance has been guided by the belief 
that school officials need concise, current information in making decisions 

4, Sections A and B' of this chapter are taken largely from Chapter I 
of The Atlanta Project: Developing Signals of Relative Performance. Bay la 
F. White, et. al .. The Urban Institute, Washington, D.C. 1972* 
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about either school system policy or day-to-day administration. Every school 
administrator, principal and teacher Is overwhelmed with numbers. The prob- 
lem has been how to convert those numbers Into meaningful Indicators of what 
Is happening In a large, complex school system. Too often, the researcher's 
response has been to create more and different numbers. But data collection 
In a large school system can be a time-consuming, expensive operation. Unless 
these new numbers are pertinent to the needs of educational declslon-makers, 
there Is no reason to collect the data at all. Therefore, the signals devised 
In this project were built on existing data, but assembled and displayed In 
a new format* 

One of the most Important operating assumptions behind the development 
of the signals has been that officials In large school systems do not know 
precisely where to look for success or failure. Fart of the problem stems 
from the absence of clearly stated goals and objectives for education, and 
agreed upon measures of success in meeting those goals. But even In the 
limited case of a school system's success in teaching basic skills, school 
officials rarely agree on where to look for specific successes or failures. 
Data used in any discussion are rarely current. Not every party to a dis- 
cussion of success or failure uses the same set of assumptions in identifying 
success or failure. Clearly, in order to know where to look for success or 
failure in teaching basic skills, there is a need for a single standard of 
measurement based on the most current data possible. 

The authors of this report believe that the measurement should relate 
to student performance, for it is data on performance that have been missing 
elements in the management of public school systems. It is technically pos- 
sible to develop a signaling system that would tell school officials which 
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school libraries have the most books or which schools have the highest pupil/ 
teacher ratios. But, unless these signals are related to how well the stu- 
dents are performing, It Is impossible to assess whether more books or more 
tenured teachers or fewer pupils per teacher are desirable. Therefore, the 
signaling system developed In this study relies on sttident performance as 
Its dependent variable. Achievement test data are presently the most readily 
and universally available Indicators of student performance in the area of 
basic skills development. Many problems exist with the interpretation of 
achievement test results. This signaling system has, however, been designed 
to reduce the effects of these problems. 

Since the signals are built on achievement test data, what should 
be the appropriate unit of observation? The signaling system focuses on 
performance in each grade within a school as the reporting unit, rather than 
the individual student or groups of students in particular projects. The 
school—which is composed of a set of grades-**i8 the basic administrative * 
unit of the school system. Although within a school the staff attempts to 
deal with the needs of individual students, management decisions about the 
allocation of educational resources inevitably involve the school or the 
grade within a school as their lowest common denominators. 

That signals should indicate relative performance is perhaps the single 
most Important assumption underlying their development. A school system has 
little control over the background characteristics of Its students. Students 
are the raw material of the educational process. A group of students enters 
school in the fall and leaves that grade, or perhaps that school, in the 
spring. It is the Job of the school system to structure its educational 
programs to the characteristics and needs of its students. Educational 

/ 
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research has shown that a strong correlation exists between the socioeconomic 
level of students and their performance on achievement tests, which Is used 
as the dependent variable In generating signals • 

Given these facts, It was deemed essential that the signals take Into 
account the characteristics of students In the school system and compare 
relative performance of schools with similar student populations. Thus, 
comparisons are not made between the very rich and the very poor. Rather, 
signals are based on comparisons of similar schools. In which performance 
Is expected, a priori , to be similar* Signals based on extremes of perform- 
ance in similar schools provide school officials with a much more precise, 
meaningful technique for pinpointing problems and successes in a large, 
diverse school system. Once problems or successes based on relative perform- 
ance are located, then the educator can make more Informed decisions about 
what action should be taken in a given situation. 

Finally, because the signals were devised as a management tool, the 
Information they provide is intended primarily for the use of school admin- 
istrators, from the principals up through the administrative structure of 
the school system. Although the classroom teacher should also be interested 
in relative performance, it was felt that signal information should be 
directed primarily toward other levels of school administration, specif- 
ically the principals and central office staffs. The classroom teacher is 
closest to the process of education and, as the primary dispenser of 
"education," already has information about how successful the process is. 
Present indicators of success or failure in the classroom grow progressively 
weaker as the distance from the classroom Increases. Decisions which 
directly affect the process of education as it occurs in the classroom are 
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often made by personnel several levels removed from the level where instruction 
takes place. Signals provide a device which enables busy school officials to 
have a better idea of what is happening in the classroom. 

B, THE BENEFITS OF A SYSTEM WHICH SIGNALS 
EXTREMES OF RELATIVE PERFORMANCE 

The potential benefits of information which signals extremes of perform- 
ance are implicit in the assumptions which underlie its development* Any 
school system has limited resources, including money, personnel, and time, 
to be used in providing education to its students, and potentially limitless 
demands on those resources. The pivotal management question is how to make 
maximum use of those resources. By giving school officials clues about what 
is happening in the schools and where > school administrators will have an 
important tool to use in making informed decisions about how and where to 
allocate those scarce resources. 

While signals can be used to locate extremes of performance, by them- 
selves they do not address the question of why the performance level exists 
or of what should be done about It. However, the information that signals 
provide can be used by school officials to diagnose problems and to prescribe 
appropriate treatments. Ideas for treating problems abound. There is a 
tendency to apply a particular remedy in shotgun fashion. A system which 
identifies schools with similar student populations can be extremely useful 
in deciding where to apply a remedy and in evaluating its success. It is 
important to remember that signals call attention to success as well as fail- 
ure, Unfortunately, educators know as little about what causes success as 
they do about what accounts for failure. Attempts to account for apparently 
successful schools or grades can help school officials narrow the range of 
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possible remedies for problems, thereby enhancing the likelihood of success 
for the remedy chosen* 

The signals flag only extremes of relative performance* The definition 
of what constitutes extreme performance has been established so as to elimi- 
nate many of the arguments about the accuracy of achievement tests In measur- 
ing Individual student achievement* The signals, by design, flag only those 
cases In which relative performance Is extremely high or low* Consequently, 
the numbers of signaled cases are more compatible with the limited resources 
available to affect those cases* 

Information from the signals Is potentially most useful In making deci- 
sions which have a direct Impact on the classroom or grade because It Is 
from the grade that the signals derive* The pattern of signals from a partic- 
ular grade and subject area, such as Ath-grade reading, can provide important 
clues for curriculum development activities in a school system* The pattern 
of performance in a particular school can be used In making decisions con- 
cerning staffing needs for that school* Patterns in achievement in particular 
types of schools can be used to identify teacher training needs or to channel 
teachers to in-service training opportunities* Thus, in the short run, 
signals of extremes of relative performance can be an important Ingredient 
in decisions about how to target flexible resources within the school system* 

In the longer run, the signals and the curves can be used to focus 
attention on important policy questions which confront school boards and 
top decision-makers within a school system* The signaling system does not 
detect net changes over time in the performance of the Atlanta School System 
as a whole* It provides a tool for use in Identifying individual situations 
that produce extremes in performance In selected grades* The shape and level 
of the fitted curves, however, raise the questions of overall system perform- 
ance and of which tyres of schools to help. One policy response might be to 
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concentrate efforts on Improving the performance of every school in which 
relative performance is extremely low. This might mean putting effort into 
a rich school where perfoimance is at or above the national norm, but below 
the performance level of similar schools in the system. Or, a policy decision 
might be made to focus attention on the schools where absolute performance Is 
lowest, regardless of the fact that some of those schools may have high levels 
of relative performance. Signals do not tell education policy makers which 
course of action is better. But the signaling system will help frame policy 
issues more clearly, and thus should raise the level of debate about the 
direction of public education. 

C. THE DERIVATION OF SIGNALS 
OF RELATIVE PERFORMANCE 

The information — signals — distributed to Atlanta officials has been 
designed to enable school personnel and administrators to Identify grades at 
individual schools where the students* performance is considerably better or 
worse than might be expected of students in similar schools. Performance is 
measured by mean (average) level on the achievement test batteries adminis- 
tered to Atlanta students in the spring of each year. Schools are identified 
as similar based on the level of student participation in the free and reduced- 
priced lunch program. 

In April of each year, Atlanta administers a battery of standardized 
achievement tests to every student in Grades 1-7.. Data on mean (average) 
achievement for the three years 1971, 1972 and 1973 have been used in the 
production of signals. For the years 1971 and 1972, signals were based on 
results of the Metropolitan Achievement Test batteries; for the year 1973, 
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signals are based on results of the Iowa Test of Baelc Skills. After con- 
sultation with Atlanta officials, only results from the reading and arithme- 
tic problem solving subtests^ were used In the Initial production of signals. 
Hie signals Identify extremes of performance relative to the economic 

composition of a school, as measured by participation In the free and reduced* 
price lunch program. Although educational research Indicates that the socio- 
economic background of students accounts for much of the observed variation 
In achievement, most school systems. Including Atlanta, do not keep accurate 
or current data on the socioeconomic status of their students. However, 
participant- Ion In the federally subsidized school lunch program can be used 
as a surrogate for a school's economic composition. Since entry Into the 
free and reduced-price lunch program Is determined by family Income and size, 
the percent of students who participate in this program provides an indicator 
of the percentage of poor students at each school* This variable alone 
accounts for 50 to 80 percent of the variation in average scores in each 
grade level, even though the variable represents the average participation 
of an entire school rather than an individual grade. 

To produce signals of relative performance, averages by grade on one 
reading and one arithmetic subtest of the achievement test battery for a given 
year ^ere plotted against the amount of participation in the free and reduced- 
price lunch program at each school. The result of this effort is one scat- 
tergnun for each grade and each subtest (7 grades x 2 subtests « 14 scatter*-* 
grami for each year). Figure II-l illustrates the relationship which 
exists between these two variables. The horizontal axis of each 
scattexgram represents the percent of students at any school participating 

TT It is possible to produce signals using the results of any achieve** 
ment subtest. 
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Figure II-l 

Relationship Between Achievement and Subsidized Lunch Participation 



+ ♦ * 



Reading Grade 5 



Low Medium^ High 

FREE AND REDUCED-PRICE LUNCH PARTICIPATION 

In the subsidized lunch program. The right end of the scattergram is 

labelled ''High" participation, indicating those schools which enroll more 

students from poorer economic backgrounds. Each mark (+) on the scattergram 

in Figure II-l represents the grade equivalent average for all students who 

f 

took the reading subtest in the 5th grade at a particular school, plotted 
against free and reduced-^prlce lunch participation for that entire school. 

Atlanta administers achievement teats In the seventh month (April) of 
each school year. The national norm for 5th graders who took the test at 
that time is defined by the test manufacturers as 5.7. It is apparent when 
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looking at the scattergram that most Sth grade average scores In Atlanta 
elementary schools fall below the national norm. The signals developed in 
this project do not compare individual school averages with the national 
norm; rather » the signals are derived from comparisons of schools In Atlanta 
which have student populations of similar economic composition* 

The process of making relative comparisons has been made easier by 
fitting a curve labelled C through the data points,^ Figure II-2 below 
shows the resulting curve for Sth grade reading in one year. Any point on 
the curve may be thought of as the norm for schools which fall above or below 
that point. Figure II-2 also Includes four other curves, two above (A and B) 
and two below (D and E) that basic fitted curve. These four curves represent 
boundaries beyond which the average performance of students in that grade is 
considerably above or below the level of average performance that might be 
expected of students in similar schools. Boundaries have been designed so 
that only clear cases of extremely high or low relative performance are sig- 
naled. Thus, only about 10-15 percent pf the grades in all schools are sig- 
naled, as cases of extreme performance, in any one year. 

Five categories of signals result from the approach illustrated in 
Figure 11-2. When the level of relative performance falls within the shaded 
range shown on Figure II-2, the grade is not considered an extreme case and 
receives no signal. A grade in which the level of performance falls between 
the curves labelled A and B is signaled with a blue semi-circle. When the 
level of performance falls on or above curve A, then the grade is signaled 
with a full blue circle. When the performance of a particular grade is rel- 
atively high, but the absolute level of performance is below the national 

norm, then the s3nnbol mB Is placed in the appropriate full blue or half -blue 

6^ The curve used here is a least squares fit. Documentation for the 
production of signals may be found in. the Appendix. 

O 
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Fig;ure II-2 
Derivation of Signals 




Low 



Medium High 
FREE AND REDUCED-PRICE LUNCH PARTICIPATION 



signal.^ When the level of performance falls between curves D and E, then 
the grade is signaled with a red semi-circle. When the level of performance 
fails on or below curve E, then th^^^ with a full red circle. 

Since only performance among similar schools is being compared, two 5th 
grades with the same absolute grade equivalent average may receive different 
signals. A 5th grade which has a grade equivalent average of 4.9 in a school 
with many poor students might be considerably above the point of the perform- 
ance curve for schools similar in terms of economic level; it would therefore 
receive a half -blue signal. However, a 5th grade which has the same grade 
equivalent average of A. 9 in a school with very few poor students might be 

~7"i This distinction between relative and absolute performance was made 
at the suggestion of former Superintendent John W. Letson. 
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considerably below the performance curve for sJjnllar schools and it vould 
receive a half-red signal. 

The signals for each elementary, primary and middle school In Atlanta 
were organized Into compact displays similar to the ones which appear on the 
following page. Figure II-3 Illustrates the Initial format for displaying 
signals for an Individual school which was distributed to Atlanta In November 
1972. The format In Figure II-4 presents an additional year of Information 
for the same school and was distributed to Atlanta officials In August 1973 » 
at the close of this project. Each set of school signals was accompanied 
by data on the mean achievement score for each grade » subtest and year. 

Presentation of data in this format enables the user to see quickly the 
relative status of performance at a school* Several important facts about 
performance in Atlanta's schools emerge from this approach to the use of 
achievement test data. First, most grades in a school are not signaled 
because the signaling system locates only extremes of performance, conserva- 
tively defined. Second, in three years of signals for approximately 130 
schools per year, there is not a single school in which every grade is sig- 
naled in both reading and math. Moreover, the pattern of relative performance 
within any school usually differs from one subtest to the other; the relative 
performance of students within the same grade on different subtests is also 
markedly dissimilar. 

Reading down a single column of one grid pictures the relative perform- 
ance of all grades in the school in reading or arithmetic at one point in 
time. Reading across a row of one grid pictures the relative performance of 
different groups of students in the same grade over time. For a school In 
which student mobility is low, reading down a diagonal compares the relative 
performance of the same group of students over time. 
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Another display of signals was prepared for use primarily by staff at 
the Area Office level* The reading and arithmetic signals for every school 
In an Area for one year were arrayed on a single sheet of paper. Staff In 
the Area Office and elsewhere throughout the central administration of the 
school system received a set of Area sheets for each year since 1971 • This 
method of displaying signals enables an administrator to see at a glance 
the relative performance of every grade in all schools (as many as 30) In 
a geographic area. 

The next section of this chapter describes In more detail some of the 
properties of sisals. 

D. PROPERTIES OF SIGNALS 

Now that the technique for the production of signals has been described » 
some questions about the properties of signals can be examined. Signals have 
been produced for three years and will be used to examine the following 
questions: 

• What proportion of grades are signaled as cases of 
extreme relative performance in each of the three 
years? 

• Are the extremes of performance which are signaled 
of significant magnitude to be considered extreme 
performance? 

0 How does the assignment of signals from year to year 
compare with that which would be expected from chance 
alone? 

• Was the observed pattern of signals produced in 1973 
different from that expected on the basis of signals 
from previous years? 

As the material which follows will show, signals do indeed indicate cases of 

extreme performance, even when measurement error is taken Ir-to accounts The 
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pattern of signals Is measurably different from the pattern which would have 
been obtained through chance alone. Moreover » for Grades 3 through 7, the 
pattern of signal change from year to year did not alter markedly after the 
signals were Introduced Into the Atlanta system. 



1. THE ANALYSIS FRAMEWORK 

In order to consider the questions posed above » an. analysis framework 

was adopted which produced a consistent set of data from year to year. A 

total of 115 schools have been Included In the analysis set. The set 

Includes only elementary schools which remained open (did not open or close) 

over the three-year period In question. The basic element selected for 

8 

analysis is the grade level (l«e., Grades 1, 2, 3, 4, 5, 6 and 7) at each 
elementary school. The analysis framework thus consists of 115 schools, 
each of which in general has seven grades for a total of 115 x 7 > 805 cells. 
Since a few schools do not contain all seven grades, tabulations will gen- 
erally be based on less than the maximum number of cells. 

Figure II-5 illustrates the analysis framework. The shaded row (School 
004) indicates the seven grade levels within a single school. The shaded 
square Is the 4th grade at School 002. Each column represents a single grade 



8. Machine grading of the test results from Grades 1 and 2 for 1973 
were subcontracted by Atlanta to the test publishers. Due to some confusion 
on format and marking, several thousand test results were not graded, ren- 
dering these two grades unusable for analysis in 1973. While there is some 
chance of recovering these data for further use, they were not available for 
use in this report. Because of this^ all data have been converted for this 
jphapter to consider only Grades 3, 4, 5, 6 and 7. Data for Grades 1 and 2 
from 1970-71 and 1971-72 have already been processed at the Institute and 
ate available. 

9* A cell is a particular grade level at a particular school. Several 
schools in the analysis set no longer have 6th or 7th grades; they have been 
transferred to middle schools. OccasionalXy, test data are missing for a 
cell, even though the grade did exist. When considering only Grades 3, 4, 
5, 6 and 7, the maximum number of cells in the analysis set is 575. 
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level for the entire axialysls set; Each cell in the figure represents a 
grade level at a particular school. This figure indicates the finest level 
of detail considered, the level to which a particular signal is assigned for 
high (blue or half blue), average (neutral), or low (red or half red) 
relative performance. For each cell, signals are created for the reading 
and the arithmetic problem solving subtests in the spring of each year. 



Figure II-^S 
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2. THE PROPORTION OF GRADES SIGNALED 

This section addresses the question of what proportion of grades are 
signaled as cases of extreme relative performance In each of the three years. 
The data presented in this section are based only on Grades 3-7, because of 
the scoring problems encountered in April 1973 for Grades 1 and 2. In each 
year since April 1971, a signal of relative performance in reading and in 
arithmetic problem solving has been produced for every grade 3-7 in the 
115 schools in the analysis set. 

A primary purpose of the signals was to help school officials locate 
cases of extreme relative performance. The limits originally used in 
developing the signals were designed to flag only about 15 percent of the 
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cells as extreme. It was felt that school officials could only respond to 

a minority of cases and that attention should be focused on the grades which 

were known to show large variation from some "norm." 

The signaling boundaries were selected in order to signal the grades at 

schools lying furthest from the curve fitted through the mean scores of all 

schools for each grade and subtest. Percentages were used because inspection 

had shown that there was generally more dispersion of scores around the curve 

for non-poor schools (where the absolute score was larger) and generally 

lower dispersion around the curve for poorer schools (where the absolute 

score was smaller). Thus» the use of a percentage boundary tended to 

equalize the numbers signaled from various economic categories. 

Table II-l shows how signals were distributed in each of the three years, 

for reading and arithmetic problem solving. "Blue" refers to either full or 

half -blue signals; "red" refers to either full or half-red signals. "Neutral" 

means the level of performance in the grade was not considered extreme and 

received no signal. 

TABLE II-l 
DISTRIBUTION OF SIGNALS FOR GRADES 3-7 





READING 


ARITHMETIC PROBLEM SOLVING 


YEAR 


Blue 


Neutral 


Red 


Total 


Blue 


Neutral 


Red 


Total 


# Z 


# X 


// % 


# 


# % 


# % 


If X 


# 


1971 


46 8.2 


480 85.7 


34 6.1 


560 


24 4.3 


518 92.5 


18 3.2 


560 


1972 


57 10.4 


454 83.1 


35 6.4 


546 


33 6.1 


500 91.7 


12 2.2 


545 


1973 


56 10.4 


453 84.3 


28 5.7 


537 


36 6.7 


493 91.8 


8 1.5 


537 



Of the cells signaled blue in reading, seven received blue sig- 
nals in each of the three years. Five additional grade cells received blue 
signals in reading in both 1971 and 1972, but not in 1973; seven other 
cells received blue signals in 1972 and 1973, but not in 1971. No grade 
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cell in Atlanta received a red signal in reading three years in a row. Six 
grade cells received red signals in 1971 and 1972; seven other grade cells 
were red in 1972 and 1973 in reading. 

Table II-2 below provides data on the amount of overlap occuring between 
reading and arithmetic signals. Each grade receiver two signals: one in 
reading and one in arithmetic. Fewer grades receive signals in arithmetic 
than in reading. 

TABLE II-2 

OVERLAP OF READING AMD ARITHMETIC SIGNALS 
FOR GRADES 3-7 



NO. OF GRADES WITH BLUE SIGNALS IN: 


NO. OF G 


RADES WITH RED SIGNALS IN: 


YEAR 


Reading 


Arithmetic 


Reading and 
Arithmetic 


Reading 


Arithmetic 


Reading and 
Arithmetic 


1971 
1972 
1973 


46 
57 
56 


24 
33 
36 


20 
19 
24 


34 
35 
28 


18 
12 
8 


11 
7 
5 



Data in the table show that a grade which receives a blue or a red 



signal in arithmetic is likely to receive the same kind of signal in reading 
about two-thirds of the time. There are, of course, many grades signaled 
(red or blue) for reading that do not receive the same signal for arithmetic* 
Further examination of actual scores would be necessary to reveal the extent 
of differences In relative performance between reading and arithmetic. 

3. HOW EXTREME IS THE PERFORMANCE SIGNALED ? 

One way to deal with the question of how extreme is the performance 
which is signaled (either red or blue) is to consider how far away from the 
basic curve each red or blue signal lies. An examination of some actual 
signaled cells will be used to illustrate what has been learned about how 
far signaled cells lie from the basic fitted curve. 
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In the process of fitting a curve through the achievement data for each 
grade, each year and each subtest, the distribution of the distance that the 
average scores fell from each fitted curve was collected. The standard error 
(CT) value for this dispersion of scores was obtained In each case. For example, 
the distance of the average score of all children In each red signaled grade 
In reading In 1972 from the fitted curve or "norm" for that grade, subtest, 
and year was measured. The variation of the average of a grade from the 
curve (expressed In grade equivalent units) was divided by the standard 
error ((t) value for all points about that curve to determine how far away 
from the curve the extreme cases tended to fall. The results showed that, 
of 35 red reading signals for 1972, 10 were more than ic below the curve. 
An additional 24 were between 1^ and 2(7* below the cur\^e. The remaining 
signal was 0.95(t below the curve; this nearest case came from a lower grade 
(3rd) at a relatively poor school (100 percent free and reduced-price lunch 
participation). These results appear to hold for the other sets of signals. 
For the signals based on arithmetic problem solving, the limits used to 
produce the signals turned out to be even more conservative. In arithmetic, 
fewer cases were signaled, and those that did receive either a blue or a 
red signal were located even further from the curve than reading signals. 

In absolute grade equivalent terms, the standard error of the distribu- 
tion of scores around the curves in 1972 ranged from 0.69 grade equivalent 
in 7th grade reading to 0.28 grade equivalent in 3rd grade problem solving. 
Percent boundaries were used in signaling because dispersion of the averages 
about the curve is not uniformly distributed and tends to decrease as percent 
free and reduced-price lunch participation increases. As an estimate of an 
average distance used in signaling, the 15 percent signaling boundaries are 
0.9 grade equivalent units from a curve at a grade equivalent score of 6.0 
and 0.A5 grade equivalent units at a grade equivalent score of 3.0. 
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The next question about the signaled cells is how accurately were their 
average scores measured. That Is, even though the boundaries were sufficiently 
conservative, what effect did measurement error have on the signal assigned to 
a particular cell? The standard error of measurement for a single student on 
the achievement tests used by Atlanta was approximately 0.4 grade equivalent 
In 3rd grade and 0.6 grade equivalent In 7th grade .^^ The average size of a 
grade ranges from 50-60 children. This suggests that the average on the grade 
Is measured within 0.06 (or 0.4/^ 50) to perhaps 0.08 (or 0.6/^50) grade 
equivalent at one standard error. It is doubtful that this small an error 
occurs in practice; however, if an error of this range did occur, it would 
not disturb the estimate that most of the cases signaled are extreme. It 
would simply mean that on remeasurement some alternative extreme cases might 
be signaled near the boundaries. 

The discussion of measurement error will become more important when 
transitions of the signals from year-to-year are considered. Up to this 
point, the discussion has considered only whether extreme cases are being 
signaled. When only signals for one year are considered, it is less impor- 
tant that some legitimately extreme performers are not signaled as long as 
those signaled are, in fact extreme. When signal changes from ye^r-to-year 
are considered, it is of Interest how many cells have average values near 
enough the boundary between extreme performance (red or blue) and average 
performance (neutral) that they could switch signals simply through measure- 
ment error. 

In the case of signal transitions from year to year, a quick test has 
been made by taking a census (through visual inspection) of those averages 
which fell near the inner 'boundary lines in 1972 reading. To estimate both 



10. See, for instance, Durast et al . , Teacher's Handbook, Metropolitan 
Achievement Tests, Harcourt, Brace and Jovanovich, New York, 1971. 
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a best and worst case, a count was made of the total number of cases (Grades 
3, 4, 5, 6 and 7) falling within +0.06 grade equivalent and +0.15 grade 
equivalent of the Inner boundary lines. The results are given In Table 
I 1-3 below. 



TABLE II-3 

TABULATION OF CASES NEAR THE BOUNDARIES IN 
1972 BEADING FOR GRADES 3-7 



BOUNDARY 


+0.06 


+0.15 


-0.06 


-0.15 


Blue/Neutral Boundary 


10 


21 


10 


26 


Red /Neutral Boundary 


11 


44 


8 


17 



Consider first the red boundary. There are eight red signaled grades 
within 0.06 below the boundary and 17 red signaled grades (including the 
previous 8) within 0.15 below the boundary, out of a total of 35 red signaled 
cells. If we postulated as a worst case that the 17 signals represented the 
la range of measured value for a larger set of signals that all lay on the 
boundary, then the total set would contain approximately 51 signals. [This 
assumes a normal distribution in which a single 1(T contains 17 signals 
and the total set, therefore, contains ^^51 signals.] Approximr "rely one- 
half of these, or about 26, would be expected to fall in the red signal 
range. One-half of the 26 (again on average) might be expected to change 
from red to neutral simply due to measurement error in the next cycle of 
measurements. That would mean that 13 of the red signals become neutral 
simply due to remeasurement. While they would still be relatively extreme, 
they would not be signaled red in the remeasurement. 

Clearly, both the error range and assumptions used have been made 
extreme in this case to show how many of the 35 red signaled cells might 
be signaled neutral during a remeasurement. Even in this worst case, the 
probability is 13/35 (or 0.37) that a red signal would become neutral. 
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If the indicated range of 0.06 had been used as la; and the same procedure 
followed, the number of potential switches would have been 6 of 35 or 17 
percent. Using the. same assumptions for blue, 7*5 (13 percent) to 16 (28 
percent) of the 57 blue signals would became neutral In a remeasurement. 
These calculations will be referred to again when transition probabilities 
are discussed later in the chapter. 

4. IS THE ASSIGNMENT OF SIGMALS DUE TO CHANCE? 

Since changes are always being made within a school system, it is 
extremely difficult to predict how much stability can be expected in the 
signals from year to year. School officials could provide no yardstick by 
which to predict the effects on relative performance of changes in teachers, 
shifts in student populations, and the continuous modification of the. 
instructional program. If all such changes were beneficial, then increasing ^ 
stability in blue signals and less stability in red signals (within the 
limits of measurement error) might be pr^'^'^lcted. The analyses conducted to 
date Indicate that most actions by schoo . v->fficials produce diverse effects. 

It is important for the reader to reali.^e that this is quite different 
from saying that no action by school officials makes a difference. It may 
be that present constraints and lack of systematic purposefulness create 
effects which cancel out. Consider the hypothetical and oversimplified 
example of a teacher-caused red signal. Suppose that the poor teacher is 
replaced with an excellent one and, in the next school year, the signal 
for that grade becomes neutral or blue. The poor teacher is not likely 
to have left the school system. Instead, the poor teacher is likely to have 
been reassigned to another school. This transaction could result in the 
creation of a new red signal (all other things being equal) . Without better 

O 

ERIC 



41 , 

attribution or more detailed analysis, the lack of systematic improvement 
throughout the school system would lead, one to conclude (as our early exam- 
inations indicate) that teacher transfer does not lead to improvement* 
The point is that this lack of effect In the statistics does not necessarily 
mean that the teachers make no difference. It can as easily mean that the 
control mechanism works in such a way that no net change is produced. 

The allocation of signals from one year to the next will now be 
examined to determine if they could have been assigned by chance alone. Most 
statistical tests address the question of whether the assignment of signals 
to cells is significantly different from a chance distribution* The data 
which follow address the following hypothesis: 

# The signals in 1972 were assigned to cells at random and were 
not conditioned upon the signal given to that cell in 1971. 

Table II-4 below contains a matrix which shows the distribution of reading 

signals for 1972 based on the reading signal received in 1971. 

TABLE II-4 

ACTUAL SIGNAIa OCCURRENCES FOR 
GRADES 1-7 IN READING, 1971 AND 1972 

1972 



SIGNAL: 


771 


Blue 


Half 
Blue 


Neutral 


Half 
Red 


Red 


32 


54 


624 


53 


8 


Blue 


15 


3 


3 


9 


0 


0 


Half Blue 


62 


8 


9 


45 


0 


0 


Neutral 


633 


19 


40 


527 


42 


5 


Half Red 


42 


2 


2 


29 


8 


1 


Red 


19 


0 


0 


1^ 


3 


2 



This matrix was divided into zones for initial testing of the hypothesis as 
follows: 
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TABLE II-5 • 
ZONE DEFINITION 



1972- 



1971 



SIGNAL: 


771 


Blue 


Half 
Blue 


Neutral 


Half 
Kftd 


Red 


32 


54 


624 


53 


8 


Blue 

Half Blue 


15 
62 


A 


B 


C' 


Neutral 


633 


f 


E 




Half Red 
Red 


42 
19 


D 


F 


G 



KEY ; A - Blue both years E - Neutral both years 

B - Blue in *71, Neutral in '72 . F - Red in '71, Neutral in '72 

C - Blue in '71, Red in '72 G - Red both years 
D - Red in '71, Blue in '72 

For the h3rpo thesis stated above to be correct, the signals in 1972 should 

be distributed proportionately among cells and any particular grouping of 

cells (such as the zones above) should not affect this,'''''' 

Consider Zone A, which is composed of cella signaled either full or 

half-blue both years • There were 77 such cells in 1971, If the signals 

were randomly and independently assigned in 1972, then the probability of 

32 54 

any cell receiving a bluef signal is — jjj^ — ■ P " 0,112, If the binomial 
distribution is used as an approximation, then the expected number of blue 
cells in Zone A is /it - np - 77 (0,112) - 8,6 and the standard deviation (a) is 
(T-^np(l-p) -\/77(0.112) (0.888) - 2,8, That is, if the hypothesis were 

true (and two possibilities in 1972 are considered; blue or not blue for the 
cells that were blue in 1971) then we would expect to find in Zone A about 
8,6 blue signals in 1972 with a standard deviation of 2^8. In fact, there are 
23 blue cells in 1972 — or about five standard deviations from the mean, 

11, The zones "neutral to blue" and "neutral to red" have not been 
carried along in this example. There are many different causes of these 
transitions and they generally occur at or near a chance rate, 

O 
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The results of similar analyses for the remaining zones are presented in 
Table II-6 (for reading) and Table II-7 (for arithmetic). The probabilities 
for each case are not shown. Instead the results are presented in units of 
standard deviations from the mean. The upper portion of each table displays 
the predicted mean (fi)f standard deviation (cr) and the observed mean (/}) for 
each zone. The lower portion of each table gives the value of Z (where Z = 
^^^ ■ ) for each zone. Where the normal is a good approximation, a unit normal 
value of Z that is a little larger than 1.6 would be representative of a one- 
tailed test at the .05 level of significance; for a two-tailed test, the .05 
level would be 1.96. The diagonal running from upper left to lower right 
represents the tendency for the cell representing the same grade to have the 
same signal in a succeeding year. The other diagonal represents the tendency 
of the signals to shift to the other extreme (e.g., red to blue, blue to red). 

On the basis of the value summarized in Tables II-6 and II-?, the 
hypothesis that signals in 1972 were assigned at random can be rejected in 
many of the zones without adjusting for other descriptive variables such as 
student mobility or staff changes. In zones A and G, the high Z values 
indicate that assignments of signals in 1972 were not random but were depen- 
dent upon the signal received in the preceding year. The positive sign on Z 
indicates that more signals were assigned in 1972 to these cells (i.e., more 
remained high or low) than would have been expected by chance. Negative 
signs Indicate that fewer occurrences existed than would have been dictated 
by chance. When examining negative Z*s, the reader will also need to examine 
fi and cr , since in some cases a nearby boundary (0 or no cases) leads to a 
value for Z that is not as large as might have been expected. 
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TABLE II-6 

VALUES FOR THE PREDICTED MEAN, THE OBSERVED MEAN, 
THE STANDARD DEVIATION, AND S-VALUES, IN READING, 
1971 AND 1972 

&. ft <r fl 



1972 



SIGNAL 


BLUE 


HALF 
BLUE 


NEUTRAL 


HALF 
RED 


RED 


TOTAL 


B 


M - 8.6 
a - 2.8 
A - 23 


M - 62.4 
<r m 3.5 

A - 54 


M - 6.1 
<r - 2.4 
A - 0 


77 


HB 


N 




M - 512.3 
<r - 9.9 
A - 527 




633 


HR 


M - 6.8 
o - 2.5 
A - 4 


M - 49.3 
(T - 3.1 
A - 43 


H - 4.8 
<r - 2.1 
A - 14 


61 


R 


TOTAL 


86 


624 


61 


771 


b. Z- Values: 

1972 




SIGNAL 


BLUE 


HALF 
BLUE 


NEUTRAL 


HALF 
RED 


BED 


TOTAL 


B 


a - 5.1 


a - -2.4 


Z - -2.5 


77 


HB 


N 




a - 1.5 




633 


HR 


8 - -1.1 


a - -1.9 


a - 4.4 


61 


R 


TOTAL 


86 


624 


61 


771 
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TABLE II-7 

VALUES FOR THE PREDICTED MEAN, THE OBSERVED MEAN, 
THE STANDARD DEVIATION, AND Z-VALUES, IN ARITHMETIC, 

1971 AND 1972 



a. /Lt or /i 

1972 



SIGNAL 


BLUE 


HALF 
BLUE 


NEUTRAL 


HALF 
RED 


RED 


TOTAL 


n 
D 


M » 4.9 
a- - 2.1 

A - n 


a- « 2.7 
A = 29 


t>- - 3.2 
<r » 1.7 
A - 2 


58 


HB 


N 




M " 566.5 
or « 9.0 
A = 600 




660 


HR 


/* - 4.4 

cr - 2.0 

A - 0 


M " 44.6 
tr - 2.5 
A - 32 


M - 2.9 
<r - 1.7 
A - 20 


52 


R 


TOTAL 


66 


661 


43 


770 


b. Z-Values: ^972 


SIGNAL 


BLUE 


HALF 
BLUE 


NEUTRAL 


HALF 
RED 


RED 


TOTAL 


B 


8 - 10.5 


a « -7.7 


a - -0.7 


58 


Hfi 


N 




z - 3.7 




660 


HR 


2 - -2.2 


z - -5.2 


a » 10.0 


52 


R 


TOTAL 


66 


661 


43 


770 
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Many celld do change signals from year to year. This can come from a 
combination of effects from teacher transfer, pupil transfer, error of 
measurement, change of teaching techniques or curriculum, etc. For Instance, 
It was observed earlier that changes of perhaps one-fourth of the signals might 
come about through measurement error. In addition^ a rather random method 
of managing changes In the educational program and faculty may produce 
apparently random results when gross statistical techniques are used. Even 
with these present, this analysis Indicates that there Is a nonrandom 
mechanic at work selecting signals. That Is, knowing the signal of a grade 
In 1971 definitely gives probabilistic Information about the signal received 
by the grade In 1972. Therefore, the hypothesis of chance assignment Is 
rejected. 

5. DID TRANSITION BEHAVIOR CHANGE 
THIS YEAR? 

As stated before, changes In the level of performance from one year to 
the next are to be expected In a school system. Students change, as do th^ 
curriculum, teaching staff, and/or materials used In a grade at a school. 
Moreover, the school system may already be taking steps to alleviate or 
replicate conditions which give rise to signals. This section addresses 
the patterns of allocation of signals of relative performance for 1971, 1972, 
and 1973 and how the transitions from year to year differ. Since data also 
were available characterizing the changes that occurred between 1971 and 1972, 
It was possible to compare the transitions In performance signals made at 
grade levels at Individual schools from spring 1971 to spring 1972 with the 
transitions made from spring 1972 to spring 1573 (the period during which 
signals were Introduced) • 
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Table II-8 presents data describing the transitions of sli^nals In 
reading for Grades 3, 4, 5, 6 and 7 of the 115-school analysis set. The table 
Includes all grades that were present and signaled In both year& (544 grades 
or cells). To understand Table II-8, think of the signals as being distrib- 
uted tP cella of grade levels at schools by the signaling technique for 1971. 
Table II-8 shows that In 1971— of the cells signaled In both yeara p44)— 44 
cells received blue signals (full blue or half blue), 467 cells received 
neutral performance signals (no color), and 33 cells received red signals 
(full or half red). Of those same 544 cells in 1972, 57 cells were signaled 
blue, 452 cells were neutral, and 35 cells were signaled red. 



TABLE II-8 

ASSIGNMENT OF READING SIGNALS, 1971 TO 1972 

1972 



SIGNAL; 


Blue 


Neutral 


Red 


Total 


Blue 


12 


32 


0 


44 


Neutral _ 


_ ^ _ - 


394 


29 




Red 


1 






33 


Total 


57 


452 


35 


544 



Observe that of the 44 cells that were blue In 1971 (top row of data), 
12 remained blue In 1972, 32 received neutral signals, and no cell became 
red In 1972. Table II-9 displays the data converted Into a probability of 
transition matrix for reading, from 1971 to 1972. The matrix shows the 
probability of making any of the possible transitions from one signal cate- 
gory to another, based on the data from 1971 and 1972. For Instance, the 
probability that a blue in 1971 would remain blue in 1972 was 0.27. 
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TABLE II-9 

TRANSITION PROBABILITIES FOR READING 
(GRADES 3-7), 1971 TO 1972 



1971 



SIGNAL: 


Blue 


Neutral 


Red 


_Blue _ _ 
Neutral 
Red 


_ 'JI — 
_.09 _ 

.03 


___.73 _ 

_ .85 

.79 


0 _ 

_.06 _ 
.18 



Consider this matrix as providing the basis for estimating the tran- 
sitions which might be expected between signals in 1972 and those which 
occurred in 1973. For instance, consider the set containing the 57 cells 
that were blue in 1972. If these 1971-72 transition probabilities were to 
apply in 1972-73, then the following distribution of signals would be 
expected for those 57 cells: 

57 X (0.27) « 15.4 would remain blue, 

57 X (0.73) "41.6 would become neutral, and 

57 X ( 0 ) > 0.0 would become red. 

The actual numbers of signals for 1973 were 14 blue, 42 neutral, and one red. 

12 

These numbers differ by less than l<r from those expected and demonstrate 

that the 1972-73 transitions of blue signals are predictable using the 

1971-72 transition probabilities for blue signals. The Implication of this 

finding is that there was no sharp change in the transition behavior of blue 

2 

signaled cells from 1972 to 1973. With these actual numbers, a simple X 

test with one degree of freedom using the actual number yields the following 

value : o 2 

X2 . (14-15.4)^ ^ (43-41.6)" ^ 

15.4 41.6 
12^^ Where a is defined as ^np (1-p) - ^57(.27) (.73)^3.3. 
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indicating again that the observed data in 1973 are distributed about as 
would have been predicted from the 1971-72 transition probabilities. 

This kind of examination was extended to cover the transition in signals 
for reading between Aorll 1972 and April 1973. Those cells from the 1972 

signal set present in the 1973 signal set must be used in developing the 

13 

expected transitions from 1972 to 1973. A total of 534 cells were present 
in both April 1972 and in April 1973. The division of the 534 cells (present 
in 1972 and 1973) into s-^.gnal categories in 1972 was as follows: 57 were 
blue in 1972, 443 were neutral in 1972 and 34 were red in 1972. The total of 
534 ceils Is slightly lower than the total in the 1971-72 transition set be- 
cause some 6th and 7th grades were transferred to middle schools in 1973 and, 
hence, were excluded from the analysis set. Applying the transition probability 
matrix to these data, the transition table for 1972/1973 would be expected 
to resemble Table II-IO* [The projections have been rounded to whole numbers 
for display here.] 

TABLE 11-10 

PREDICTED TRANSITIONS OF READING SIGNALS 
IN 1973, BASED UPON 1972 DATA 



SIGNAL: 


Blue 


Neutral 


Red 


Total 


Blue 


15 


42 


0 


57 


Neutral 


40 


376 


27 


443 


Red 


1 


27 


6 


34 


Total 


56 


445 


33 


534 



Table ll-ll then compares the observed transition counts with those pre- 
dicted for 1972-73. The largest variation is In the case of neutral to red 

[there are fewer reds observed than were predicted] ; this is not 

iJ] Data for Grades 1 and 2 were not yet available for 1973 and there- 
fore the number of cells is reduced to 534. 
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statistically significant. For the school system as a whole, therefore, 

there is no indication of a large change in transition behavior in reading 

between the two periods 1971-72 and 1972-73. 

TABLE 11-11 

COMPARISON OF PREDICTED/OBSERVED 
TRANSITIONS OF READING SIGNALS 
BETWEEN 1972 AND 1973 



1972 



SIGNAL: 


Blue 


Neutral 


Red 


Blue 








Neutral 




376, 
^383 


'20 


Red 


\ 


"/~~ ~ 
'26 


~'7 



A similar examination can be made for arithmetic problem solving signal 
transitions. Here, lower numbers of red and blue signals make the results a 
little harder to rely on statistically. 

Table 11-12 presents outcomes for arithmetic similar to that pre-* 
viously reviewed for reading. The probability of transition for the 1971-72 
case has been used to predict outcomes in 1973. The predicted distribution 
(rounded to whole numbers) is shown in Table 11-12. 

TABLE 11-12 

PREDICTED TRANSITIONS OF ARITHMETIC SIGNALS 
IN 1973, BASED UPON 1972 DATA 



SIGNAL: 


Blue 


Neutral 


Red 


Total 


Blue 


13 


19 


1 


33 


Neutral 


24 


459 


5 


488 


Red 


0 


9 


3 


12 


Total 


37 


487 


9 


533 
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Table 11-13 displays the comparison of the predicted transitions for 
arithmetic signals to the observed signals. In this case, there Is some- 
what more variation than In reading. Both blue-to-blue and red-to-red 

show smaller populations than predicted. The difference in the blue dlstrl- 

2 

butlon would be statistically significant ^ - 3.25 with 1 degree of free- 
dom) aL the 0.1 level, but not at the 0.05 level of significance. That is, 
there are fewer blues remaining blue than would have been predicted from 
1971-72 data. 

TABLE 11-13 

COMPARISON OF PREDICTED/OBSERVED 
TRANSITIONS OF ARITHMETIC SIGNALS 
BETWEEN 1972 AND 1973 



SIGNAL: 


Blue 


Neutral 


Red 


Blue 


'B 


'25 


_ V 


Neutral 




459, 
'454 




1 Red 


\ 




\ 



There are also fewer reds remaining red than might have been predicted, 
although not statistically significant (the tables contain rounded data and 
the actual expected values are red-to-neutral - 9.4 and red-to-red ■ 2.6, 
where a binomial approximation of <r -^npq » ^ 12(.22) (.78) -«^l.<i) . Since 
the reduction is also somewhat symmetrical, one might also wish to examine 
the residual error at each point, if any significant behavior conditioned on 
signals had been found that might affect problem-solving signals in this way. 
Since no signal-conditioned actions were found that were likely to have 
caused this change, the behavior .may be a function of the particular 
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signaling technique In use on these data, or a function of test differences, 
or a reflection of the myriad of changes taking place In a fairly unsystematic 
way within the school system. 

6. CONCLUSION 

These above Illustrations display some of the properties and 
behavior of the operational signaling system in producing signals of cases 
of extreme relative performance. The signal curves Illustrate but do not 
directly address some additional Important policy questions raised by the 
distributions of achievement scores within the Atlanta School System. For 
example, the shape of the fitted curve shows that poorer schools tend to have 
Increasingly lower achievement scores. That this Is generally true In school 
systems has been documented by extensive national research. Whether lower 
scores In poorer schools are acceptable and whether methods are available to 
alter this condition In a local school system are policy questions of some 
magnitude. Signals provide no answer to these policy questions. Rather, the 
signaling technique merely points out that this condition seems to recur year 
after year. 

What the signaling technique does adjust for Is the economic level of 
the students at each school and. In essence. It creates a new "norm" (the 
fitted curve) for all schools based upon the economic level of the students 
at each school. Once this Is done. It Is seen that some grades at some 
schools still exhibit high or low performance relative to other schools with 
similar student populations. Signaling Isolates these cases of high and low 

14^ There Is some Indication that the mathematical problem solving 
results may be distributed somewhat differently on the Metropolitan Achieve- 
ment Tests than on the Iowa tests. The Department of Health, Education and 
Welfare had not released the results of Its "anchor test" study of test 
comparisons In time for their use In this report. Later work may be able 
to consider this. 
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relative performance so that they are brought to the attention o£ the school 
system. That Is, signaling provides short-term Information: the locations 
In the school system where some extremes of relative performance are occur- 
ring after an adjustment has been made for the socio-economic characteristics 
of the student population. 

The next question for consideration Is how the relationships between 
signals and the decisions and actions of school officials were established. 
The next chapter explains the approach used In this study. 
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CHAPTER III 
THE EXPERIMENT: SETTING AND PROCEDURES 

The signals, described In the preceding chapter, were distributed to 
Atlanta school officials in November 1972. The purpose of this chapter is 
to acquaint the reader with the Atlanta School System, to give the reader a 
perspective on the major events of the 1972-73 school year, and to describe 
the procedures used to monitor the activities of school officials in order to 
detect the impact of signal information. 

A. CHARACTERISTICS AND STRUCTURE 
OF THE ATLANTA SYSTEM 

When the schools opened in the fall of 1972, the Atlanta system had an 
enrollment of over 96,000 students, a teaching staff of 4,700 and an annual 
General Fund budget (excluding capital funds and most federal funds) in 
excess of $97 millions. Enrollment, which peaked several years ago, had 
declined to 91,000 by the end of the school year. Black students accounted 
for over 75 percent of the total enrollment in 1972-73, and the proportion 
continues to grow* 

Atlanta students typically attend elementary school (Kindergarten through 
Grade 7) and high school (Grades 8-12). However, there has been movement 

15. In September 1973, enrollment had declined to 89,000 students (ap- 
proximately 80 percent black); the teaching staff had declined to 4,600 
positions. 
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toward a new organizational pattern which will divide students into elemen- 
tary (K-5), middle (Grades 6-8) afid high schools (Grades 9-12). During 
1972-73, there were 123 elementary schools, 5 middle schools and 25 high 
schools organized into five geographic areas. As total enrollment decreases, 
the total number of schools and the size of the teaching staff also decrease. 

Figure III-l illustrates the organizational structure of the Atlanta 
system during the year under study. Once every four years, voters in Atlanta 
elect a Board of Education to set general policy for the school system, in 
close consultation with the Superintendent and his staff. The School Board 
has authority to raise revenues for the operation of the schools; no public 
approval is required for changes in the property tax rate. 

The School Board appoints the Superintendent and the other principal 
administrative officers of the system. The Superintendent of Schools pre- 
sides over the day-to-day operations of the schools, aided by a staff of 
six Assistant Superintendents, a Comptroller and five Area Superintendents. 
Unlike many school systems, Atlanta has had no Deputy Superintendent. Flex- 
ibility has been the key organizing principle of the school system. The 
major officials of the school system have operated in thi^ absence of any 
written definitions of their responsibilities and functions. 

In general, the Assistant Superintendents and the Comptroller function 
as staff to the Superintendent. With one exception, each has been respon- 
sible for certain central activities associated with the operation of the 
schools. Each Assistant Superintendent provides services to schools and 
teachers which are usually filtered through the Area Office. Each Area 
Superintendent serves as a line administrator, responsible for all aspects 
of the educational process occurring in the schools in his geographic area. 
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What follows Is a capsule description of the responsibilities of each major 
administrator and/or division within the Atlanta system as they existed In 
1972-73. 



1. Assistant Superintendent for Administrative Services . This division 
provides centralized services to the schools and to other parts of 
the central administrative structure. Major responsibilities Include: 
purchasing and distribution of expendable supplies, materials, fur- 
niture, textbooks and library materials; maintenance of school build- 
ings and grounds; operation of food programs; provision of school 
detectives to Investigate crimes which occur on school property; 
maintenance of pupil records on attendance, enrollment, and mobility. 

2. Comptroller . The Comptroller Is responsible for the formulation of 
Atlanta's annual General Fund budget as well as for the supervision 
of all payroll and accounting activities which accompany the adoption 
of the budget. Atlanta has adopted a program budget format for the 
preparation and accounting of Its funds. The computer facility Is 
part of the Comptroller's responsibilities. Pupil attendance, prop- 
erty Inventories, payroll, textbook, library and audio-visual supply 
purchasing, some grade reporting , high school student scheduling and 
achievement test processing are all computerized In Atlanta. 

3. Assistant Superintendent for Instruction . The Instructional Division 
Is the largest of the staff groups In the Atlanta structure. Al- 
though the division has about two dozen different components. Its 
activities can be grouped Into four broad categories: the develop- 
ment, maintenance and revision of curriculum; teacher training and 
staff development; the administration of special projects or programs 
(usually federally funded); and pupil services. Including the admin- 
istration of activities for exceptional children, guidance, health, 
and social services. 

4. Assistant Superintendent for Personnel . This division Is responsible 
for the recruitment, certification and placement of all school system 
personnel. Personnel directs the competitive process established 
for selecting Individuals to fill any position above the level of 
the classroom teacher. Substitute teachers are recruited and cer- 
tified by Personnel (although the Individual school principal Is 
responsible for arranging to have a particular substitute cover an 
empty classroom). 

5. Assistant Superintendent for Research & Development . A primary 
function of the Research & Development unit Is the development of 
new federal projects for the school system. Staff members prepare 
proposals and, once federal funding Is secured, have responsibility 
for federally required evaluation of those projects. R&D coordinates 
all research projects underway in the school system. Staff members 
produce a newsletter, distributed throughout the school system, sum- 
marizing the results of selected educational research. 
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6. Assistant Superintendent for School Plant Planning ^ This division 
Is responsible for the acquisition of land for future facilities, 
the planning and construction of all new facilities and major ad- 
ditions to existing facilities, major renovations, and for the 
equipping and furnishing of new buildings. Money for these activ- 
ities comes from a separate Capital Funds budget. Most construction 
Is done under contract to the school system. 

7. Area Superintendents . The five Area Superintendents seirve as the 
direct link between the Superintendent, the reet of his staff and 
the schools. Each Area Superintendent has a staff of Instructional 
personnel, called Resource Teachers, who observe, assist and help 
train classroom teachers In the schools In that Area. The Area 
Superintendent figures prominently In every decision which affects 
the operation of a school; organization and staffing; recruitment 
and assignment of principals and teachers; selection of schools as 
sites for special programs; equipping or remodeling schools; assign- 
ment of pupils; curriculum selection; budget preparations; community 
relations, etc. 



B. THE YEAR IN ATLANTA 

The Introduction of Information about relative performance was not the 
only event to disturb the normal routine of the school system during the 
1972-73 school year. In fact, 1972-73 witnessed major changes both In the 
organization of the school system and In the personnel responsible for Its 
day-to-day operation. 

Throughout the entire school year, the Atlanta School System was Involved 
In a school desegregation suit whose settlement affected the asslgxunent of 
students and staff and resulted In a major upheaval In the administrative 
structure and personnel of the school system. Between October 7, 1972, when 
the Appellate Court first ordered the school system to come up with an ac- 
ceptable desegregation plan, and August 23, 1973, when that same Court finally 
upheld the plan (agreed upon by the plaintiff and the Atlanta School System 
in February 1973), for a period of nearly one year, the Atlanta school 
community was in a state of uncertainty about the scope and nature of any 
future arrangements. 
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Under the terms of the compromise plan finally accepted by the Court, 
no Atlanta school would have less than 20 percent black enrollment, although 
some all*<black schools would remain. The required racial composition of 
Atlanta schools was to be achieved through a combination of techniques, In- 
cluding pairing of schools, voluntary student transfers, school closings and 
some busing. In 27 schools, teachers would be reassigned to alter the racial 
composition of the faculty. Over 2/3 of Atlanta's schools had been affected 
by the compromise plan by the time school opened In September 1973. 

The compromise plan did not stop with the transfer of students and staff. 
A major feature of the compromise plan was the provision that at least 50 
percent of the top administrative positions, including that of Superintendent, 
were to be filled by blacks. To accomplish this, 16 new administrative posts 
were to be created. The search for a new Superintendent began in the. middle 
of the school year and continued until July 1973, when Dr. Alonzo A. Crlm 
was named to succeed Dr. John W. Letson. The 16 new positions remained vacant 
and their responsibilities undefined, pending the arrival of the new Super* 
Intendent. The process of selecting a new Superintendent, coupled with the 
potential expansion and reorganization of the top-level administrative staff, 
created an atmosphere of uncertainty among existing administrative personnel 
which resulted in a general reluctance to make decisions or to take actions. 
To a large extent, the school system marked time during much of the year. 

Quite Independent of the desegregation plan, a number of other changesi 
in key administrative personnel occurred during the year. For several 
years preceding 1972-73, none of the top 13 executive positions in the school 
system changed hands. Yet, within a 12-inonth period, beginning in August 1972, 
five of the executive staff, including the Superintendent, three Area Super- 
intendents and the Assistant Superintendent for Personnel, were replaced. 
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Even the School Board was not limnune from change during 1972-73. In 
March 1973, the portions of the Atlanta city charter governing the method of 
election to the School Board were revised. Effective as of the October 1973 
election, the Atlanta School System will be governed by a nine, rather than 
a ten member School Board. The method of election of Board members was 
changed so that several of the then sitting Board members would have to step 
down or seek re-election from the same district • 

As mentioned earlier in this chapter, total student enrollment has 
declined in recent years. Moreover, there have been important shifts among 
the existing population. As a result, the distribution of schools and 
students among the five geographic Areas had become very unequal. In October 
1972, the School Board adopted a plan to divide the system into six Areas, 
each serving approximately 15,500 students and 25 schools. Because of the 
uncertainty over the desegregation plan, that reorganization plan was aban- 
doned. Once the school closings and shifts of pupils necessitated by the 
desegregation plan were agreed upon, a new plan for organizing the school 
system was drawn up. The new plan, put into effect at the close of the 
1972-73 school year, equalizes the number of schools and students in each 
of five Areas. 

The administrative reorganization and the desegregation plan have had 
far-reaching effects on students and staff. Within the space of one school 
year, the following changes occurred: 12 schools were closed and 4 were 
opened; 2 high schools were converted to middle schools; 24 schools were 
affected by Area boundary changes. Alternative arrangements had to be made 
for students and teachers whose schools were closed. For those schools 
affected by the new Area boundaries, the new school year would meau a new 
Area Superintendent, a new set of Resource Teachers and, in general, becoming 
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accustomed to a different style of administration* For Area Office staff, 
the changes have meant becoming familiar with the program, staff and problems 
of new schools* 

The disruption resulting from the Court decision and the reorganization 
was compounded still more by a significant reduction in federal funds avail- 
able to Atlanta under Title IV-A of the Social Security Act. In December 
1972, Atlanta's share of Title IV-*A funds for the next 12 months was reduced 
by about 80 percent (an amount equivalent to about 10 percent of the General 
Fund budget). The funds had been used to provide early childhood education 
programs, after-school programs for older children and a variety of supple- 
mentary services. The school system had hired staff under one-year contracts 
at the opening of school, on the assumption that Title IV-A funds would con- 
tinue to be available at the 1972 level. The drastic reduction in IV-A funds 
for 1973 forced school system officials to close down programs in mid-year 
and to find other employment for IV-A staff with teaching contracts. 

In sum, during 1972-73 a number of major changes occurred in the Atlanta 
School System to disrupt the normal activities of school personnel; these 
changes did affect the continuity of school system activities. However, no 
school system is ever static. 

C. THE SCOPE AND STRUCTURE OF THIS STUDY 

The research effort which is the subject of this report began in November 
1972 with the distribution of signal information to Atlanta school officials. 
The objective of the study was to determine what effect (if any) the pre- 
viously unavailable information on relative performance had on the manage- 
ment of the school system. The hypothesis which was the subject of 
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Investigation can be stated as follows; 

Information on relative performance, when introduced into 
a sufficiently well-understood planning and management structure, 
does (or does not) produce measurable changes (a) in the decision- 
making process of a local school system, (b) in the decisions made 
by school officials, and (c) ultimately, in school performance. 

In its simplest form, the question of whether the signals had an effect 
on school system management involves an analysis of the information feedback 
loop within the school system. Figure III-2 represents that simple feedback 
loop . 

FIGURE III-2 
SIMPLE FEEDBACK LOOP 



School System 1 SIGNALS 
Structure \ 




Educational 
Process 




Measures of 
Students and 
Performance 




Comparisons 

and 
Analysis 


1 


- — • 



The school system has an administrative structure which is responsible for 
the operations of the educational process. That process occurs in a set of 
schools and classrooms which comprise tlie school system. Signals are de- 
rived from measures which define the composition of the schools and the 
performance of the students in those schools. Comparisons are made from 
these measurements and the results analyzed and synthesized to produce a set 
of signals which are then fed back into the schools* administrative structure. 
In theory, if school system officials respond to the signals and alter their 
O , actions, the resulting changes should affect subsequent measurements, 
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comparisons and signals. The cycle Is then repeated* 

In an actual school system, of course, the process Is not that simple* 
The administrative structure of a large school system involves several layers, 
each with different responsibilities and differing degrees of influence over 
the educational process. Each administrative unit may have need of different 
information to support the decisions it can make. The process of education 
occurs in hundreds of classrooms throughout the system. Each school has a 
different set (or sets) of students and an educational program that may vary 
substantially from grade to grade. A basic purpose of the school system is 
to make changes, over both the long and the short term, in the knowledge and 
behavior of its students. Measurements of educational process, the students, 
and eir performance are, at best. Imperfect and often non-existent. And, 
instt^ud of one feedback loop, there will be many — to each administrative 
level in the school system, to the school board- to the general public, to 
parents. Each party to a decision brings pressure to bear on the educational 
process in response to a variety of incentives. The question under study has 
been the extent to which signals of relative performance became an incentive 
to school system officials to change the educational process. 

Any assessment of the Impact of information on the operations of a large, 
complex organization is both difficult and tlme-constiming. It was necessary 
to limit this one-year research effort to manageable proportions by focusing 
on those portions of the school system originally thought to have some im- 
mediate, direct Impact on the grade or classroom where the measure of relative 
performance is taken. Based on earlier work in Atlanta, it had become 



16. White, et al ., op. cit . 
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apparent that the introduction of signal infomation would not have an equal 
impact on the operations of all parts of the school system. While it could 
be argued that all school system officials should have an interest in infor- 
mation about relative school performance, not all units within the administra- 
tive structure had an equal opportunity to affect the process of education 
in the classroom in the short time space of this study* 

Two criteria were used to select the administrative units to be studied 
in the effort to discover the effects of signal information on the school 
system's operations: (1) there must be some direct interaction between the 
administrative unit and the classroom; and (2) the administrative unit must 
have some flexibility to react to signal information. The first criterion 
reflects the fact that the signals are derived from the school and its com- 
ponent grades or classrooms. Each signal reflects the relative level of 
performance of the students in a particular grade at a particular point in 
time, rather than the level of performance of a group of students in an 
entire school or in a special program (e,g.. Title I) or special project 
(e,g,, after-school study centers). Some administrative units clearly have 
more direct links to the classroom than others. The second criterion, the 
flexibility to act, also stems from the nature of the signals. Signals 
represent clues to the existence of problems and successes at a particular 
point in time. Less than one school year elapsed between the distribution 
of 1972 signals and the production of 1973 signals. Some administrative 
decisions or practices could not be expected to change in so short a period 
of time. 

Applying these two criteria to the Atlanta administrative structure, 
four units of the school system seemed most likely to be affected by infor- 
mation about relative school performance: the Area Superintendents and their 
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staffs, the Divisions of Instruction and Personnel, and the principal. Each 
of these administrative units has direct and sometimes daily contact with the 
educational process at the classroom or grade level. These units work singly 
and often in concert to improve that process. Yet, there was little evidence 
that the performance of students had been a major consideration in decisions 
governing most of the activities of any of the administrative units under 
study • 

The principal, the Area Superintendents and their staffs, and the Divisions 
of Instruction and Personnel engage in a variety of activities which may 
affect the educational process. In its simplest form, the components of 
that process can be thought of as students in a classroom, an instructional 
program and a teacher or teachers • This study considers those activities 
which can affect the process from tht outside: (1) changing the teacher 
or teachers; (2) changing the content or the materials used in instruction; 
(3) augmenting the skills of the teacher (s). Educational practitioners 
(and the authors of this report) make the assumption that changes in staff, 
the instructional program or the skills of the staff can improve the per- 
formance of students in the classroom. And, since the level of relative 
performance of a grade in a school is the basis for the signals, the ac- 
tivities of principals. Area Superintendents and their staffs, and the In- 
struction and Personnel Division which may change the staff or the skills 
of staff or the instructional program in a grade have been the subject of 
this inquiry. In each ca&e, the study team attempted to determine how the 
activities are "normally" carried on by school officials and how signal 
information affected those activities. The following paragraphs pose some 
of the questions about the impact of signals. 
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1. Effects on the Recruitment » Assignment and Reassignment of Staff . 
Decisions about school staffing Involve principals. Area Super- 
intendents, and the Peraonnel Division. Prior to the Introduction 
of signals, decisions about the recruiting and placement of new 
teachers did not appear to be Influenced by data on student per- 
formance In the grade where a vacancy existed. Neither did student 
performance appear to enter Into decisions about the reassignment 
of existing staff or the composition of the staff at a school or 

in a grade. Would signal Information affect decisions about the 
desired skills or characteristics of new teachers, the placement 
of new teachers, the reassignment of existing staff, the staffing 
of an entire school? 

2. Effects on the Instructional Program , After consultation with 
teachers, principals. Area Office staff and members of the Instruc- 
tion Division, changes may be made in the structure, content or 
materials of the instructional program in a grade. Are signals 
used by any of these parties in decisions about where to make 
changes in the instructional program or in deciding which changes 
to make? Do staff of the Instruction Division or the Area Office 
attempt to associate extremes of relative student performance with 
particular textbooks, instructional approaches or organizational 
arrangements? 

3 . Effects on Efforts to Improve the Skills of Teachers , Improvement 
in the skills of teachers can be made in several ways: through 
the provision of direct assistance to the teacher by another member 
of the school system's staff or through the teacher's participation 
In in-service training programs. Area Resource Teachers, who are 
curriculum specialists assigned to Area Offices, regularly provide 
direct assistance to classroom teachers and also conduct workshops 
and supervise other in-service activities. The Instruction Division 
also organizes in-service programs. Do signals Influence the way 

in which Resource Teachers allocate the time or effort they spend 
in direct assistance to teachers or the decision about which 
teachers to assist? Are signals a factor in decisions by principals 
or Resource Teachers to refer teachers to in-service training 
courses? Are signals used In decisions about the subjects of In- 
service programs? 

Once the administrative units of primary Interest to this study were 
identified, efforts tfere made to determine the extent to which the new infor- 
mation (signals of relative performance) was utilized in the completion of 
assigned responsibilities. In the short run, there could be at least two 
different kinds of impact or effect. The new Information could have had 
a direct effect on the actions of one or another part of the school system. 
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For example, as a result of the signals, a principal might have done some- 
thing to treat the situation revealed by the signals. But the new Information 
might have had little or no direct effect on action and still had an effect 
on the deliberations which precede the decision to act. In this latter case, 
the observed pattern of action will have remained unchanged, but the signals 
might have been consulted by the parties to the action. In the longer run, 
the cumulative effect of using new Information In thinking about a course 
of action should be to affect the action taken. Since this study focused 
on a time span of less than one year. It has been concerned with the effects 
of signal Information both on actions taken by school officials and on their 
deliberations , whether or not any action was taken. 

As stated earlier, this project Is predicated on the twin assumptions 
that the activities and actors selected for study have some potential for 
affecting the educational performance of students and that those activities 
might be affected by signal Information. The truth of the first assumption 
remains to be demonstrated because the Interactions described In this study 
are not well documented either In the research literature or, more Importantly, 
In the Atlanta School System. In other words, there Is neither an accepted, 
recognized level of Interaction between any two parts of the school system, 
nor a clear relationship between Increases In the quality or quantity of the 
Interaction and Increases In student performance. 

, The absence of any advance knowledge about the level or content of 
"normal" activities selected for study has had Important Implications for 
testing the effects of signals. First, considerable effort was devoted to 
preparing a description of these routine activities, without regard to signal 
Information. Second, It was clearly Impossible to specify In advance what 
level of change would occur In any activity after the Introduction of signals. 

O 
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Therefore, the approach taken has been to see if grades which received signals 
Indicating relatively high or relatively low performance were given dispro- 
portionate attention by school officials. 

The next section of this chapter explains the procedures used to gather 
data during the study. 

D. PROCEDURES USED IN THE STUDY 

Two different techniques were used to gather new data for the study: 
structured Interviews with participants In the activities under examination 
and special forms or surveys on which Atlanta personnel recorded Information 
about specific activities. Efforts to learn about the school level of the 
Atlanta School System focused on a randomly selected sample of 37 elementary 
schools. 

1, STRUCTURED INTERVIEWS 

Table III-l (see next page) provides an Indication of the scope and 
content of the Interviews conducted during the course of this study. Prin- 
cipals, Area Superintendents and Resource Teachers were Interviewed on two 
different occasions. 

In several Instances, similar questions were asked of the parties to 
an activity or decision which might affect the classroom or grade to provide 
a picture of how each viewed the seirvlce rendered or the decision made. In- 
formation about the activities of Resource Teachers, Area Superintendents 
and the Division of Instruction collected during 1972-73 was augmented by 
less structured interviews in an earlier phase of this project. 
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TABLE III-l 
STRUCTUBED INTERVIEWS 



DATS OF 
INTERVIEW 


PERSONS 
INTERVIEWED 


FOCUS OF INTERVIEW 


December 72/ 
January 73 


5 Area Superintendents 


Reactions to signals; explanation 
of specific signals; anticipa- 
ted response to signals. 


August 73 


4 Area Superintendents 


Actions taken since Introduction 
of signals. 


February 73 


31 Resource Teachers 


Reactions to signals; explanation 
of specific signals; descriptions 
of the nature and extent of Re- 
source Teacher contacts with 
teachers, principals. Area Su- 
perintendents, Instruction Di- 
vision staff, and other parts of 
the school system. 


June 73 


18 Resource Teachers 


Whether and how Resource Teachers 
had responded to signal Infor- 
mation ^ review of Resource 
Teacher assistance to classroom 
teachers. 


January - 
February 73 


37 Principals 


Reactions to signals; Information 
aoouc ,cne genc&cix o&gamzaibiou 
of the school. Its staff, socio- 
economic composition; contacts 
between the school and other 
parts of the school system. 


May 73 


37 Principals 


Specific actions taken as a re- 
sult of signals; services pro- 
vided to the school by Resource 
Teachers, Instruction Division 
staff; changes In Instructlcfial 
program and staff for the coming 
school year. 


January - 
Augtiat 73 


3 Personnel Staff 


Information about the process of 
staffing schools and the effects 
of signals on that process. 


May 73. 


6 Instruction Division 
Staff 


Information about services pro- 
vided to schools and the pro- 
cedures for making changes In 
curriculum or textbooks. 
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2. REPORTED DATA 

Three new data collection Instruments were devised and used in this 
study. Each is described below. 

a. REPORT OF SCHOOL VISIT FORM 

The Report of School Visit form (Figure III-3) was designed for use 
by a Resource Teacher each time he/she visited a school to assist a teacher. 
The form provided the first uniform information about which schools and which 
teachers received assistance from Resource Teachers > as well as other impor- 
tant information about the nature of the assistance provided. The Resource 
Teacher indicated on the form the status of the teacher visited (tenured or 
probationary), school and grade visited, the primary subject in which assist-* 
ance was given, length of the visit, who Initiated the visit, the services 
rendered during the visit and the diagnosis or recommendation made. The 
forms were printed in triplicate, so that two copies could be retained in 
the Area Office for use there. 

The forms were used regularly by Resource Teachers in all five Areas 
from October 1972 until the end of the school year. There were 83 Resource 
Teachers assigned to Area Offices In 1972-73 » Including both curriculum 
specialists (60) and social workers, psychologists and visiting teachers (23). 
The forms were Intended for use primarily by curriculum specialists. Table 
III-2 provides Information about the use of the Report of School Visit form 
in each Area. Since no comparable data about this facet of Resource Teacher 
activity existed prior to this time, it is difficult to determine the extent 
to which the data provided by Resource Teachers accurately portrays their 
activity. However, the variation in the total number of forms completed by 
individual Resource Teachers, both within and amiong the Areas, may indicate 
thiat Resource Teacher visits to schools were under-reported. 
O 

ERIC 



72 



FIGURE III-3 

ATUNTA PUBLIC SCHOOLS 

Report of School Visit 



Check Check 



Resource Teacher's (DO NOT WRITE IN DARK BOXES) 
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School 



n M II 



Date of visit. 



Primary subject in which assistance was given (Check one) 



ED 



PI I M 



. reading oi 

. math 02 

. language arts 03 



. science 04 
social studies os 
.art/music oe 



. foreign language 07 
.physical education os 
.vocational education 09 



10 



discipline 

organization 11 

planning 12 



Place an X on the line below to indicate the approximate length in hours of this visit (excluding travel time ). 



L 



1 



1 



1 



1 



4V4 



1 



5Vfe 



1. Who initiated this visit? (Check all that apply) 

a. You d. Area Superintendent 

b. Classroom teacher e. Other (specify: 

c. Principal 

2. What services did you render during this visit? (Check all that apply' 
a. Observation of the teacher 

b. Demonstration of teaching technique, materials or equipment 

c. Consultation or conference 

d. Assistance with or interpretation of tests 

e. Assistance with extracurricular project 

f. Other (specify: 



3. What recommendation did you make? (Check all that apply) 

a. Recommend workshop or in-service program 

b. Recommend proficiency modules 

c. Follow-up visit scheduled for 

. d. No follow-up necessary 



. e. Problem referred to another Resource Teacher, 
.f. Other (specify: 



nn 



f 1 1 1 1 1 1 



PI 1 1 1 1 1 



4. Additional comments on the visit. 
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TABLE III-2 

RESOURCE TEACHER RESPONSE TO REPORT OF SCHOOL VISIT FORM 

BY AREA, 1972-73 





AREA 
1 


AREA 
11 


' ABEA 
I III 


' AREA 
I IV 


AREA 
' V 


TOTAL 


Ihsmber of Resource 
Teachers in Area 


20 


20 


1 U 
1 


1 16 

1 


1 16 


83 


Number of Resource* 
Teachers Returning Form 


13 


8 


1 9 
1 


1 12 
1 


' 10 


52 


Number of Forms Returned* 


672 


221 


' 1012 

1 


1 1010 


266 


3181 


Fewest Forms Completed* 
by Any Resource Teacher 

Host Forms Completed by 
Any Resource Teacher* 


' 1 
1 


76 


1 " 

■1 

1 294 


1 2 
1 

1 335 

1 


1 2 
' 74 





* Numbers exclude visits reported to secondary schools since 
the analyses of how signal Information affected Resource 
teacher visits are confined tc those grades for which 
signals existed (I.e.. Grades 1-7). 



b, REPORT OF WORKSHOP OR IN-SERVICE PROGRAM FORM 

The Report of Workshop on In-Service Program form was also designed 
for use by Resource Teachers. It provides information — again, for the first 
time — about the effort Resource Teachers devote to in-service training activ- 
ities during the year. The Report of Workshop or In-Service Program form 
(Figure III-4) was completed by Resource Teachers each time that they prepared 
and/ or conducted a workshop or in-service program. The Resource Teacher re- 
corded information about who prepared the program, how much time was spent in 
preparation, who conducted It, the subject and objectives of the program, its 
length of time and who attended it. Table III-3 provides Information about 
the number of workshop forms completed during the 1972-73 school year. 
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FIGURE III-4 
ATLANTA PUBLIC SCHOOLS 

Report of Workshop or In-service Program 

Area Ha tP nf t h i «; p rngra m 

Program prepared by: Program conducted by: 

rnntiniiPri from. (Hatp) 

Will hp nontiniJPri nn- (Hatp) 

Timp of this program? from \ty 

Siihjprt 

nhjprtiups 

Total time spent in preparation for this program hours 
Names of participants at the program 

NAME SCHOOL GRADE/TITLE 


Do not 
write in 
this space 
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TABLE IH-3 

RESOURCE TEACHER RESPONSE TO REPORT OF 
WORKSHOP OR IN-SERVICE PROGRAM FORM, 
BY ABEA» 1972-73 





Am I ABEA lABEA i ASM. 1 AKU 
I II III IV 1 V 


TOTAL 


Mo. of forms completed 

No. of Basource Teachers 
coiVletlas forae 


70 1 0 ' 87 ' 33 1 14 
12 t 0 ^ 7 ^ 10 ^7 

1 t. 1 


204 
36 



c. CLASSROOM TEACHER SURVEY 

A mall survey was sent to a sample of teachers In 37 elementary 
schools selected for intensive study. The survey was designed to gather 
Information about the kinds of services ^Ich were actually received by 
classroom teachers In 1972-73. Such Information would provide a picture of 
how the teacher perceived the services provided by other parts of the system 
and would help to corroborate Information gathered In other parts of the 
study. « 

The 37 elementary schools were composed of 199 grade levels or cells. 
Teachers in 96 of those cells were Included In the survey: A8 cells were 
selected at random and 48 cells were selected to provide additional Infor- 
mation about the services provided to teachers of grades where performance 
was relatively high or low. A total of 233 teachers received questionnaires 
in May 1973; each questionnaire was accompanied by a letter explaining its 
purpose and assuring the respondent's anonymity. 

Table III-A shows for each Area the number of teachers sampled and the 
number and proportion of questionnaires returned. 
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TABLE III-4 
RESPONSE TO TEACHER QUESTIONNAIRE, BY AREA 



AREA 


Numbsr 
Sampled 


Number 
Returned 


Percentage 
Returned 


I 


45 


31 


69 


II 


46 


34 


74 


III 


53 


26 


49 


IV 


52 


35 


67 


V 


37 


28 


76 


TOTAL 


233 


154 


66 



The next two chapters of this report describe what was learned as 
a result of these data collection activities. 
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CHAPTER IV 

THE MANAGEMENT OF LARGE URBAN SCHOOL SYSTEMS— 
AS OBSERVED IN ATLANTA 

One of the results of this study has been the accumulation of knowledge 
about the actual operations of a large, urban school system. This chapter 
presents the information in terms of a discussion of each of the three deci- 
sion areas chosen for study: staffing, the instructional program and improv- 
ing the skills of staff. 

A. INTRODUCTION 

This study is concerned with the effects of information on the manage-* 
ment of a large school system. It is based on the assumption that actions 
of school officials which directly affect the educational process will ulti-- 
mately affect the performance of students in the classroom. 

The preceding chapter Identified the parts of the school system adminis- 
trative structure and the three activities which have been chosen for ^tudy. 
The tasks addressed by this chapter are (a) to define for those activities 
how decisions are made and how the various administrative parts fit together 
and (b) to identify the existing bases for making decisions and the incen- 
tives which exist for or against change. An understanding of these tasks 
will, in part, account for the way Atlanta officials responded, or failed 
to respond, to signal Information. 

Most of the material in this chapter was collected during the 1972-73 
school year. Each set of activities which will be discussed In the remainder 
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of this chapter Is assumed to have a direct effect on the level of perform- 
ance in the classroom or grade — and, thus» an effect on the signals of 
relative performance. 

B. THE RECRUITMENT. ASSIGNMEKT 
AND REASSIGNMENT OF STAFF 

There are two basic sources of teachers for any classroom: the teaching 
staff already employed by the school system and the pool of applicants for 
teaching positions. Decisions about which teacher or teachers stand before 
a class may involve the principal* the Area Superintendent and the Personnel 
DlviiBlon. Personnel Division staff take the lead In recruiting ner? teachers 
and in making the initial assignment of each new teacher; the Area Superin- 
tendent plays the central role in the transfer or reassignment of existing 
teachers. The interactions which occur between the Personnel Division » 
the Area Superintendent and the principal when a new teacher is hired or 
a teaching assignment is changed are described below. 

1. RECRUITMENT OF NEW TEACHERS 

The process of recruiting potential teachers for Atlanta schools operates 
somewhat independently of the actual need for teachers in the classroom. Per- 
sonnel Division staff direct the recruiting drlve» f jpported by a recruitment 
committee conqposed of representatives from all parts of the school system. 
Most recruiting has centered on college campuses; as a result, most new teach- 
ers hired by Atlanta have been recent college graduates. 
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The annual cycle of hiring and assigning new teachers occurs as follows: 

• Throughout the school year: a file of candidates is created by 
accepting and screening applications from prospective teachers. 
Applicants complete a standard written application form, which is 
supplemented by a personal interview* 

• Late spring and summer: "blanket contracts"^^ are offered to 
candidates based on decisions about the number and type of new 
teachers required for the subsequent school year, 

• Late summer and early fall: vacancies which cannot be filled 
with existing teachers are filled by new teachers; additional 
teachers are hired and assigned as needed* 

Ordinarily, vacancies which have not been fi^\ed by the reassignment 
of existing staff are filled by new teachers. The request for a new teacher 
originates with the principal, who identifies the grade and school where the 
vacancy has occurred and the race needed to maintain the required racial 
composition of the faculty. The principal sends the request to the Area 
Superintendent, who reviews it. If approved, the request 16 forwarded to 
Personnel, Xn addition to information about the location, grade and the 
race of the position. Personnel may have available information on the Instruc-* 
tional approach or organizational pattern used in the school. While official 
information available to aid placement is limited, principals and Area 
Superintendents do communicate to Personnel information about the vacancy 
on an informal basis. 

Once the assignment is made. Personnel informs the teacher, principal 

and Area Superintendent, If the teacher accepts the assignment, the Personnel 

Division considers the vacancy "filled" and does not participate further in 

the transaction. In only a very few cases is the new teacher hired with 

knowledge of the grade and school to which he or she will be assigned. The 

rk'incipal may receive formal notification that a new teacher has been assigned 

17^ A "blanket contract" is a contract to teach in the Atlanta School 
System with the specific assignment to be made later. 
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to the school after the teacher has arrived. Principals have no advance 
Information about the characteristics of the r^ew teacher or the teacher's 
Instructional strengths and weaknesses. 

A}.1 new teachers are required to participate in an orientation program. 
Half of the program is designed and conducted by the Area Office staff and 
half by the central teacher training staff of the Instruction Division. . 

A new teacher is on probation for a period of three yc^ars. At the end 
of each of those years, the principal must appraise the teacher's performance 
and make a recommendation about whether the teacher is to be retained for 
another year. The Area Superintendent reviews the principal's rating and 
recommendation. Before supporting n recommendation not to retain a pro- 
bationary teacher, the Area Superintendent usually makes an Independent 
appraisal of the teacher, either directly or by sending a Resource Teacher 
to observe the teacher's performance. 

Since total student enrollment has been declining for the past several 
years, the need for new teachers has resulted primarily from resignations 
and retirements of existing staff. Resignations are submitted by the teacher 
directly to the Superintendent, with copies to the principal and Area Super- 
intendent. For the 1970-71 school year, an extensive recruiting effort was 
required to fill the large number of vacancies which occurred following the 
court-ordered adjustment of the distribution of black and white teachers at 
each school In the Atlanta system. By September of 1970, approximately 
1,000 new teachers were offered Jobs for the school year — nearly doublv'i the 
usual number. In September 1.972, the school system hired and placed 324 
elementary school teachers. Due to the rapidly declining enrollment and 
the closing of 12 schools, no new elementary school teachers wer6 Mred 
prior to the opening of school in September 1973. 
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2. ASSIGNMENT OF EXISTING STAFF 

During the course of employment by the Atlanta School System, a teacher 
may be transferred to a new grade assignment, transferred to a new school 
(in the same Area or In another Area) or may be promoted to another position 
within the school system. 

a. TRANSFERS WITHIN A SCHOOL 

In Atlanta, as in nearly all school systems, the most common and 
basic pool of personnel consists of the staff at an individual school. Once 
assigned to a school, a teacher is under the immediate supervision of the 
principal. Typically, the final assignment of teachers to specific grades, 
classes and tasks is made by the principal. In particular, principals can 
and do alter grade assignments of teachers in the school from school year to 

school year ox, in some cases, during the school year. These transfers can 

« 

be motivated by any of the following: teacher dissatisfaction with an assign* 
ment; suggestions by the Area Superintendent; decisions to alter the staffing 
pattern and instructional program in the school. The Area Superintendent la 
always consulted about a change, but Personnel Is not involved in these decl* 
sions. The Personnel Division is notified of a change In grade assignments, 
but there may be a considerable time lag. 

b. TRANSFERS AMONG SCHOOLS 

Each spring, the principal and Area Superintendent meet to review 
the school's instructional program, staffing needs and budget for the coming 
year* Based on the student enrollment projected for the following September, 
the staffing allocation for the school is established. The combined staffing 
allocation for all schools in an Area form a somewhat self-contained personnel 
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pool. As long as an Area Superintendent malntalna acceptable student/teacher 
and black/white staff ratios at each school, he can redistribute personnel 
assigned to schools In the Area at his discretion; he need not seek the 
approval of Personnel or of other Area Superintendents. 

Staff laay be transferred Into a school for any or all of the following 
reasons: Increasing enrollment at the school; resignations or retirements of 
existing staff; changes In the organization or Instructional program at the 
school; special needs of the students; to meet the required racial compo- 
sition of the faculty. At their annual spring staffing conferences , the 
principal and Area Superintendent review the type of personnel needed at the 
school. The Area Superintendent looks first to the pool of teachers within 
his Area In making assignments to a school. It Is only when the Area Super- 
intendent cannot fill the positions by transferring staff within his Area 
that the other Area Superintendents and the Personnel Division are called 
Into the staffing process. 

Principals have relatively little input Into the decision about which 
teacher (s) will be transferred into their schools. If a principal knows a 
specific teacher is available and that the teacher has the necessary attri- 
butes for the Job, the Area Superintendent may help in getting that teacher 
assigned to the school. In most cases, a teacher is transferred into the 
school without the advice or consent of the principal. The principals inter- 
viewed expressed dissatisfaction with the way in trhlch these transfers take 
place. At present, the formal notice of the arrival of a new teacher comes 
from Personnel; the notice gives only the name and previous school assignment 
of the teacher. Principals expressed their preference for more information 
about the new teacher in advance of the teacher's arrival at the school. 
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Requests for transfer may be dent directly to the Area Superintendent 
by the teacher; such requests do not need the approval of the principal. A 
request for transfer Is sent on to Personnel by the Area Superintendent. 
The teacher may Indicate his/her preferences for reassignment. Administra- 
tive transfers (those due to school closings or thj elimination of positions 
at a school) have priority over requests for reassignment due to personal 
reasons. The Personnel Division takes an tiCtlve role in the reassignment 
of teachers only when the transfer Involves more than one Area. Then Person- 
nel compiles a list, ordered in terms of seniority, of all teachers requesting 
a transfer and distributes the list to all Area Superintendents. A teacher 
who is transferred can appeal the transfer directly to Personnel. 

3. SUMMARY 

During the past 12 months, four factors have governed the staffing 
decisions of Atlanta officials. Decreasing student enrollment has had 

an obvious Impact on staffing. No new teachers, except special education 
teachers- have been hired since January 1973; Atlanta now has a huge waiting 
list of applicants for teaching positions. The closing of 12 elementary 
schools and the conversion of two high schools into middle schools is the 
second factor affecting decisions about staff assignments. Teachercf from 
closed schools are given priority in new assignments; high school teachers 
are not always qualified to ti^ch in middle school grades. The third major 
factor affecting staffing has been the drastic cutback in federal funds (es- 
pecially Title IV-A). Teachers hired under annual contracts had to be 
absorbed by the system when federal funds were cut in mid-year. Atlanta 
has not had to let any teachers go thus far. The natural attrition of staff 
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due to retirements and resignations has made the situation somewhat less 
desperate. The major factor governing staffing assignments for 1973-74 
is the need to meet the requirements of the court-ordered racial ratios 
in individual schools. 



C. SHAPING THE INSTRUCTIONAL PROGRAM 

Decisions affecting the instructional program (the second component of 
the educational process at the classroom level) are also made at several 
levels within the Atlanta system. The details of the instruction program 
for a particular grade or school are the primary concern of the principal 
and staff at the school. The development and revision of curricu'^um for 
the entire system are a major concern of the Division of Instruction. The 
Implementation and day-to-day review of certain facets of the instructional 
program in the classroom fall to the staff of the Area Office. Each unit 
works with the others, even on those activities for which one unit assumes 
primary responsibility. The next two sections describe the shaping of the 
instructional program from the perspective of a school and of the school 
system as a whole. 

1. SHAPING THE INSTRUCTIONAL PROGRAM 
FOR A PARTICULAR GRADE OR SCHOOL 

Atlanta principals claim major responsibility for decisions about the 
way grades or classrooms are organized and the instructional materials and 
approach used throughout the school. The principal will discuss and seek 
approval for changes In organization or instructional approach with the 
Area Superintendent. Most principals tend to share responsibility for 
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decisions about organizational or curriculum changes with the teaching fac- 
ulty* The choice of textbooks is often made by vote of the entire faculty 
oc the faculty of the grade(s) involved* Over the past few years, there has 
been a shift from the traditional self-*contained classrooms toward more flex- 
ible teaching arrangements. Once a different organizational pattern is 
adopted. Area Resource Teachers are usually called upon to provide assistance 
to the school (principal and teachers) in implementing the new pattern* 

The principal and teachers can and do seek advice on Instructional 
matters from both the Instruction Division staff and Area Resource Teachers. 
The Instruction Division sometimes sends Information to the school concerning . 
new text offerings in various subject areas; curriculum guides are created 
to assist teachers; Information is available about texts and materials; 
periodic workshops are sponsored by the Instruction Division for principals 
and teachers* Principals have indicated that advice on the choice of text- 
books is the most sought after service provided by the Instruction Division* 
While curriculum coordinators from the Instruction Division do visit schools, 
principals interviewed In this study claim overwhelmingly to Initiate those 
visits* The two exceptions to this statement are those schools where there 
are federal projects (e.g., Follow-Throu«h or Title I) operating and those 
schools piloting the new elementary curriculum. Direct involvement in the 
instructional program at a school by Instruction Division staff is more fre- 
quent in these instances. 

A c^nplaint frequently expressed by principals is that there is no 
routine ^fay in Atlanta to share information about good practices which exist 
in a school or schools. The principals feel that the existing mechanisms 
for exchanging Information about a particular method or text or innovation 
at a school or by a particular teacher are inadequate. The Division of 
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Research and Development does sunsnarlze and disseminate information about 
research findings > hut the presentation may be too general or the results 
not timely encngh for the needs of the principal. While the Area Superin- 
tendents have reguler meetings of all principals, even these meetings do not 
provide an appropriate fonns to exchange ideas for several reasons. First, 
the meetings include principals of all schools — primary, elementary, middle, 
and high schools — and the innovation may be at such a fine level of detail 
that only a few of those present would be interested, ferbaps more impor*- 
tantly, the agenda for these meetings is often fixed, with no place on it 
for a general discussion of innovative ideas or practices. Within the 
school, the principal may encourage teachers to observe the classes of 
colleagues who have demonstrated instructional success. But principals 
repeatedly expressed the need for a regular, systematic way to share the 
information beyond the school level. 

2. SHAPING THE INSTRnCTIONAL PROGRAM 
FOR THE SCHOOL SYSTEM 

Within the broad outlines of an instructional program for the entire 

system, each school attempts to tailor ite specific educational program to 

the needs of its students. The Division of Instruction, which takes the lead 

in molding that program* Is the largest of the staff groups in the Atlanta 

structure. The development of curriculum and the administration of special 

16 * 

projects (usually federally funded) are two of its major activities, which 
bear directly on the educational process in the classroom. 

18? The other broad categories are teacher training (discussed in the 
next section of this chapter) and pupil services. TJiese categories are 
arbitrary groupings used by the authors of this report. The Instruction 
Division has about two dozen components, each headed by a director or coor-» 
dinator. It has had no formal organizational structure. 

O 
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In the Atlanta system, the development or revision of curriculiim is a 
Joint undertaking, rather than the exclusive responsibility of the Curriculum 
Development department • The entire elementary curriculum is undergoing revis- 
ion by a committee composed of teachers, principals. Area Superintendents and 
Area Kepource Teachers, as well as staff from the Instruction Division. 
According to a 1970 progress report, the Elementary Curriculum Revision Com*-* 
mittee was given the task of developing a curriculiim model which "would enable 
a teacher to begin teaching any child at his performance level in a given area 
and help him move forward." The model was to include: 

« provisions for individualizing instruction (analysis and diagnosis, 
appropriate content, teaching strategies, and evaluation) 

• provision for the child to perform adequately on his own level 
(self-pacing, self-evaluation, and self-direction) 

• plans for continuous progress of students 

• provision for the development of self-understanding and under- 
standing of others. IS 

The revised cux^rlculum was Introduced on a pilot basis at one school 
in 1971-72* During the period covered in this study, the revised curriculum 
was in use at nine more schools* Hi^r.h school which adopted the new curric- 
ulum was provided with a manual containing i:>formation on how to implement 
the curriculum, teaching aids (such as the teaching ccodel) , as well as . 
technical assistance from the Curriculum Revision Committee's etaff . In 
addition to technical assistance provided directly from the Instruction 
Division, one Resource Teacher in each Area was responsible for assisting 
the elementary schools piloting the new curriculum and for obtaining equip- 
ment and/or materials needed to Implement it. In the administration of 

19. "Elementary Curriculum Revision Committee Progress Report," 
Atlanta Public Schools, Summer 1970, p. 1. 
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the school with the new curriculum, there Is a hieavy dependence on community 
Involvement and iolnt decision making among principal, teachers and parents. 
The ten pilot schools were selected because It was believed that the prin- 
cipal^, staffs and parents would be receptive to the new curriculum and 
methods of organization. 

Another major project of the Instruction Division Is the Comprehensive 
Instructional Program (GIF), a. locally funded compensatory education project 
designs ^ to Improve the basic skills of elementary students. The GIF program 
was begun three years ago. During Its first tt^ years, It concentrated on 
the teaching of reading In the primary grades; last year the focus was 
expanded to Include the te^^jhlng of arithmetic In upper elementary grades. 
The two tangible products of the GIF program to date have been the develop-* 
^nt of diagnostic r'Sadlng tests for use In Identifying and correcting 

reading problems of Individual students and the development of proficiency 

20 

modules for use by elementary school teachers of reading and math. No 
special diagnostic test for arithmetic has been developed. 

The mechanism for Implementing the GIF program has been the assignment 
of 19 Resource Teachers to the five Area Offices to assist principals and 
teachers In Improving the teaching of reading and arithmetic. However, once 
Resource ^wachers were assigned to the Area Offices, It was left to the 
Area Super ^udent to decide how the Resource Teachers would be distributed 
among the scliools In the Area. The director of the GIF program and the 
Resource Teachers meet weekly to discuss Issues of common concern. 



20« Froflclency modules will be discussed In more detail in the 
teacher training section of this chapter. 
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Staff of the Instruction Division, known as subject-area coordinators 
or specialists, are available to individual schools or teachers for consult- 
ation on curricular matters* However, F.ost of the curriculum coordinators 
are oriented toward middle and high schools. One way the Instruction Divi- 
sion does have a direct effect on elementary schools is in the selection 
and purchase of textbooks. Most textbooks are paid for with state monies 
and their adoption is subject to state approval. According to state guide- 
lines, a textbook may have a life of no more than five years* Once every 
five years, the state reviews all texts in a given subject or discipline 
and produces a list of textbook series that can be used in the state of 
Georgia. Following the publication of one of these lists, the Atlanta Text- 
book Coordinator establishes a committee of teachers. Resource Teachers, 
and Instruction Division staff to review and rate texts on the basis of 
their appropriateness for use with students performing at various levels. 
The results of these reviews (about a paragraph on each text series) are 
distributed to the principal as a basis for selecting text series for use 
in the school. Typically, one text series in a given discipline is adopted 
for an entire school. 

3. SUMMARY 

Decisions about the shape and content of the Instructional program are 
probably the best example of the fluid management which characterizes the 
operation of Atlanta schools. Each instructional decision usually involves 
one or more representatives of each administrative level in the Atlanta 
system. Educational goals are vaguely worded. The outlines of the instruc- 
tional programs to meet those goals are filled in under the direction of 
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the Instruction Division staff. This staff, however, spends relatively 
little time actually In the classroom. Instead, the Instruction Division 
looks to Area Resource Teachers to make operational those guidelines. There 
appears to be a lack of any clear delineation of the responsibilities 
of Area Superintendents and their staffs on the one hand and the Instruction 
Division on the other, for the Implementation of the Instructional program. 
As a result, the principal and teachers at a school make the crucial deci- 
sions about the actual way In which the curriculum Is assembled and imple* 
mented. No well-defined feedback mechanism exists for staff outside the school 
to determine what kind of Instructional program really exists In a classroom 
and what problems the teacher may encounter In Implementing the curriculum. 

D. IMPROVING THE SKILLS 
OF THE TEACHING STAFF 

A third set of actions which affect the educational process In the 
classroom are those which attempt to Improve the skills of the teaching staff. 
Two kinds of actions will be considered: (1) the provision of direct assist- 
ance to classroom teachers, which usually occurs on a one-to-one basis; and 
(2) the participation by the teacher In one of the structured teacher train- 
ing activities available In Atlanta. 

« 

1. DIRECT ASSISTANCE TO TEACHIERS 

Classroom teachers can and do initiate requests for direct assistance. 
Often they turn to colleagues on the staff or to the principal. However, 
because the principal and the teacher have daily contact within the school, 
the principal is often the person who first becomes aware that a classroom 
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teacher needs assistance. According to principals Interviewed In this 
study, a major clue tc a teacher *s performance comes from the principal's 
personal observation of the teacher. Most principals report several visits 
a week to each classroom to check on both teacher and student progress. 
Casual visits to the classroom are augmented by periodic conferences between 
the principal and teacher. The principal may also demonstrate a lesson for 
a teacher or teachers; 40 percent of the principals Interviewed reported 
giving demonstration lessons on such topics as reading, how to use diagnos- 
tic tests, science and story telling. 

Outside the school, the most readily available source of assistance 
for the classroom teacher is the Area Resource Teacher. Sixty of the 83 
Area Resource Teachers are curriculxim specialists; the number of curriculum 
specialists assigned to any one Area varies from 6 to 15. Math, science, 
Bngllsh, foreign language and social studies Resource Teachers are assigned 
to work primarily with high school teachers and principals. Art, music and 
library Resource Teachers work with art or music teachers or librarians 
assigned to the schools in the Area, and only infrequently with classroom 
teachers directly. The remainder of the curricula specialists in each 
Area work almost exclusively with elementary schools. 

The way in which the Resource Teachers are utilized varies from Area 
to Area. Some Area Superintendents assign each elementary Resource Teacher 
to a group of schools (although the ntimber of schools assigned varies from 
Resource Teacher to Resource Teacher, even within the same Area). Decisions 
about which teachers to assist, what services to provide or the conditions 
under which a visit is made are usually left to the discretion of the 
Resource Teacher. 
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21 

During 1972-73, Resource Teachers reported 3181 visits to elementary 
or middle schools. Each time a visit was made, the Resource Teacher was 
asked to Indicate who Initiated the visit: the Resource Teacher, the teacher 
the principal, the Area Superintendent* Table IV-1 below presents the respon 
ses to that question [multiple responses were permittee*.], tabulated by Area. 



TABLE IV-1 

INITIATORS OF RESOURCE TEACHER VISITS, 1972-73* 



Initiator 


Total 
(n-3181) 


Area I 
(n-672) 


Area II 
(n-221) 


Area III 
(n-1012) 


Area IV 
(n-1010) 


Area V 
(n-266) 


Resource 
Teacher 


64% 


63% 


69% 


49% 


77% 


68% 


Teacher 


23% 


30% 


23% 


20% 


20% 


27% 


Principal 


19% 


23% 


11% 


22% 


16% 


19% 


Area Super- 
intendent 


12% 


27% 


2% 


9% 


11% 


0.4% 



Source: Report of School Visit Forms 



According to the Information provided by Resource Teachers, half to three- 
quarters of all visits to elementary or middle schools are Initiated at 
least In part by Resource Teachers. Teachers are credited with Initiating 
visits more often than principals. 

Resource Teachers may provide assistance to teachers In a variety of 
currlcular or Instructional areas. During 1972-7?, Resource Teachers were 
asked to Indicate, for each visit reported » the primary subject of the 
assistance given. Twelve options were provided: nine related to currlcular 
areas; the other three were planning, organization and discipline. Although 
asked to Indicate only one subject for each visit, most Resource Teachers 

21. See Chapter III p« 70 ff • for an explanation of the uiie of the 
Report of School Visit Form. 
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reported providing assistance on several topics during a single visit. 
Table IV-2 below sunasarizes data on the subject of the assistance reported 
during 1972-73. 



TABLE IV-2 

SUBJECT OF THE ASSISTANCE GIVEN 
BY RESOURCE TEACHERS, 1972-73 



SUBJECT OF 
ASSISTANCE 


Total 
Cn-3181) 


Area I 
(n-672) 


Area II 
(n-221) 


Area III 
(n-1012) 


Area IV 
(n-lOlO) 


Area V 
(n»266) 


Reading/ 
Language Arts 


47X 


64% 


29% 


24% 


55% 


82% 


Arithmetic 


10% 


13% 


1% 


4% 


14% 


16% 


Social Studies 


5% 


2% 


13% 


8% 


3% 


3% 


Science 


42 


6% 


2% 


2% 


5% 


2% 


Planning/ 
Organization 


36% 


44% 


16% 


42% 


27% 


43% 


Discipline 


3% 


6% 


3% 


2% 


3% 


4% 



* Source: Report of School Visit Forms 



Reading and language arts have been the most frequent subject of Resource 
Teacher visits , except in Area III. The frequency with which assistance was 
provided In other subject areas drops off sharply. Planning and organiza- 
tion seem to have been the other major concerns during 1972-73. 

An examination of data on the number of schools and grades visited at 
least once during the year provides another insight into the pattern of 
assistance provided to classroom teachers. Based on reports filed by 
Resource Teachers , at least one classroom teacher (K**7) was visited in all 
but four of the 127 elementary and middle schools during 1972-73; in addi- 
tion. Resource Teachers reported visits to principals and other staff not 
assigned directly to the classroom (e.g.» music teachers) In more than 85 
percent of the schools. 
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The number of grades visited at least once during 1972-73 is shown in 
Table IV-3 below. The data are presented by Area, first for Grades l'-7 and 
then for kindergarten* Based on reports filed by Resource Teachers, teachers 
in 60 percent of the 946 grades in Atlanta elementary and middle schools 
were visited at least once during the year. There ^rae considerable varia«- 
tion among the Areas in the coverage reported by Resource Teachers. 



TABLE IV-3 

PROPORTION OF GRADES 
VISITED BY RESOURCE TEACHERS, 1972-73* 



Area 


Grades 
Total 1% 


1-7 
Visited 


Kindergarten 
Total \X Visited 


Grades K-7 
Total 1% Visited 


I 


126 1 


75% 


19 


1 89% 


145 


1 77% 


II 


154 1 


41Z 


22 


1 77% 


176 


1 45% 


III 


179 1 


62% 


27 


1 44% 


206 


1 60% 


IV 


189 1 


71X 


27 


1 33% 


216 


1 67% 
1 51% 


V 


175 1 


58Z 


28 


. 11% 


203 


Total 


823 1 


61% 


123 


1 47% 


946 


, 60% 



^Source: Report of School Visit Forms 



Perhaps a better Indication of the extent of the direct assistance 
provided by Resource Teachers can be seen in Table IV**4 below. The table 
shows the frequency with which Resource Teachers visited individual grades 
(l-*7) in individtial schools. While frequency does not say anything about 
the length of the visits or the nature of the assistance provided, it dees 
indicate the amount of exposure Atlanta teachers had to Resource Teacher 
assistance in the classroom. Thirty percent of 506 grades (l*-7) visited 
were visited only once during the year; 15 percent were visited only twice 
by Resource Teachers; just 8 percent of the grades visited were visited 
more than ten times. The pattern of visits by Area IV Resource Teachers 
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TABLE IV-4 

FREQUENCY OF VISITS TO GRADES 1-7 
BY RESOURCE TEACHERS, 1972-73* 



Percent of Grades 1-7 Visited 



Frequency of 
Visits 


Total 
(n-506) 


Area I 
(n-95) 


Area II 
(n-63) 


Area III 
(n-112) 


Area IV 
(n-135) 


Area V 
(n-101) 


1 


29% 


29% 


48% 


31% 


14% 


36% 


2 


15 


8 


21 


16 


11 


20 


3 


12 


. 10 


11 


12 


14 


13 


4 


10 


13 


2 


10 


10 


12 


5 


6 


7 


10 


4 


7 


4 


6 


6 


4 


6 


5 


6 


8 


7 


4 


2 


3 


5 


7 


4 


8 


3 


6 


0 


4 


4 


2 


9 


3 


7 


0 


0 


6 


1 


10 


2 


2 


0 


4 


4 


0 


Over 10 


8 


9 


0 


8 


17 


1 



* Source: Report of School Visit Forme 



la somewhat different. In that Area, 17 percent of the grades visited were 
visited on more than ten occasions, and only 14 percent of the grades visited 
were visited only once. 

The pattern of assistance to classroom teachers varies from Resource 
Teacher to Resource Teacher. Table IV**5 (see next page) presents Informa- 
tion about the visits reported by each of the 52 Resource Teschers who 
supplied Information for 1972-73. The designations I-A, I-B, etc., identify 
individual Resource Teachers who filed reports* Column 1 indicates the 
total number of elementary and middle schools visited by that person, either 
to aid a classroom teacher or to visit with some other member of the school's 
staff. Colvimn 2 Indicates the number of those schools for which at least 
one report of assistance to a teacher in Grades 1-7 was made. Column 3 shows 
the number of different teachers (in Grades 1-7) assisted by each Resource 
Q '^eacher during the year, 
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TABLE IV-5 

SCHOOLS VISITED AND CLASSROOM TEACHERS 
ASSISTED, BY RESOURCE TEACHER, 1972-73* 



RESOURCE 
TEACHER 
BY AREA 


COLUMN 1 


COLUMN 2 


COLUMN 


3 




TOTAL NUMBER 
OF SCHOOLS 
VISITED 


SCHOOLS WHERE 
ASSISTANCE WAS 
PROVIDED TO TEACHERS 
IN GRADES 1-7 


TOTAL NUMBER OF 
TEACHERS ASSISTED 
IN GRADES 1-7 


I-A 


.18 


6 


7 






I-B 


15 


12 


19 






I-C 


8 


0 


0 






I-D 


4 


0 


0 






I-E 


5 


5 


39 






I-F 


2 


2 


31 






I-G 


11 


9 








I-H 


12 


3 


5 






I-I 


6 


4 


5 






I-J 


7 


5 


34 






I-K 


5 


0 


0 






I-L 


10 


0 


0 






I-M 


3 


3 


17 






II-A 


4 


4 


4 






II-B 


11 


2 


4 






II-C 


8 


7 


1 7 
1/ 






II-D 


5 


4 


12 






II-E 


19 


19 


47 






II-F 


5 


4 


18 






I I-G 


19 


3 


3 






XX— n 


A 


0 


0 






III-A 


27 


25 


72 






III-B 


28 


15 


44 






III-C 


22 


10 


46 






III-D 


22 


11 


20 






III-E 


26 


19 


48 






III-F 


13 


4 


6 






III-G 


29 


0 


0 






III-H 


24 


13 


21 






III-I 


11 


1 


1 




— ... 


IV-A 


14 


13 


28 






XV— o 


Q 
O 




1 






IV-C 


19 


16 


35 






IV-D 


10 


10 


79 






IV-E 


5 


5 


45 






IV-F 


8 


7 


82 






IV-G 


2 


2 


4 






IV-H 


5 


5 


24 






IV- 1 


1 


0 


0 






IV-J 


3 


3 


8 






IV-K 


12 


0 


0 






IV-L 


2 


0 


0 




V-A 


5 


3 


7 






V-B 


4 


4 


14 






V-C 


9 


2 


2 






V-D 


4 


4 


11 






V-E 


2 


2 


2 






V-F 


4 


4 


44 






V-G 


13 


9 


35 






V-H 


4 


4 


26 






V-I 


4 


4 


39 






V-J 


2 


0 


0 
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As can be seen in Table IV-5» the nusaber of schools visited by a single 
Resotirce Teacher ranged from 29 to 1; the ntonber of teachers In Grades 1-7 
assisted by a single Resource Teacher ranged from 82 to 0. Two-thirds of 
the 52 Resource Teachers reporting visited at least one school in which no 
classroom teacher was assisted. Variations within Areas are as great as 
variation between Areas. 

22 

The reports filed by Resource Teachers accounted for 3»625 hours 
spent visiting the schools. Table IV-6 shows the total amount of time each 
Resource Teacher reported (Col. 1) and the amount of time spent with class- 
room teachers in Grades 1-7 (Col. 2). The final column In Table IV-6 shows 
the proportion of reported time spent with teachers In Grades 1-7. The 
remaining portion of each Resource Teacher's reported time was spont with 
prlncipalst kindergarten teachers and other staff not directly assigned to a 
grade. Although 52 Resource Teachers reported visits to schools » ten of 
them worked exclusively with school staff not assigned directly to Grades 1-7. 

In summary » according to the reports filed by Kesource Teachers » the 
direct assistance provided by them during 1972-73 tended to be primarily self- 
initiated. Reading/langtiage arts and planning/organization were the primary 
subjects of assistance rendered. Resource Teachers visited teachers in 61 
percent of the elementary grades at least once during the year. Fifty-six 
percent of the grades which were visited at all by Resource Teachers were 
visited three times or less during the year. The number of schools visited t 
the number of teachers assisted » the amount of time spent in direct assist- 
ance varies from Resource Teacher to Resource Teacher. 

22. This total probably underrepresents the time sptnt at schools 
for three reasons: the forms w^re not used during the first month of school; 
Resource Teachers do not appear to have reported every visit to a school; no 
time was reported on approximately 3 percent of the completed forms. 

ERLC 
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TABLE IV-6 

TIME SPENT AT SCHOOLS VISITED » 
BY RESOURCE TEACHER» 1972-73* 



RESOURCE 






COLUMN 3 


XV XAw X xnb v X d nviu cvo / 


TCfTkL TIME {IN HOURS) 


PERCENT OF TOTAL 


TEACHER 


RFPOllTin) SPBNT 
n£»«uAXCti/ D«cinx 


REPORTED SPENT 


REPORTED TIME 


BY AREA 


AT ^mnnT Q 

AX OwuUUwO 


IN GRADES 1-7 


SPENT IN GRADES 1-7 


I-A 


148 


28 


19 


I-B 


42 


25 


60 


I-C 


73 


0 


0 


I-D 


70 


0 


0 


X-E 


184 


136 


74 


I-F 


22 


22 


100 


I-G 


89 


54 


61 


X-H 


66 


6 


10 


I-I 


9 


6 


66 


I-J 


128 


90 


70 


I-K 


8 


0 


0 


I-L 


40 


0 


0 


X-M 


24 


24 


100 


XI-A 


4 


4 


100 


II-B 


44 


8 


18 


II-C 


56 


49 


87 


II-D 


24 


20 


85 


II-E 


64 


60 


93 


TT— T? 


32 


26 


82 


I I-G 


40 


3 


8 


II-H 


15 


0 


0 


III-A 


124 


75 


60 


III-B 


302 


39 


13 


III-C 


150 


83 


55 


III-D 


77 


40 


52 


III-E 


160 


69 


43 


III-F 


46 


7 


15 


III-G 


55 


0 


0 


III-H 


86 


52 


60 


III-I 


25 


2 


8 


IV-A 


60 


58 


97 


IV-B 


14 


0.5 


4 


IV-C 


101 


83 


82 


IV-D 


313 


267 


85 


IV-E 


107 


98 


91 


IV-F 


376 


339 


90 


IV-G 


7 


7 


100 


IV-H 


36 


36 


100 


IV-I 


6 


0 


0 


IV-J 


36 


32 


90 


IV-K 


46 


0 


0 


IV-L 


2 


0 


0 


V-A 


7 


6 


86 


V-B 


22 


22 


100 


V-C 


32 


8 


25 


V-D 


15 


13 


87 


V-E 


2 


2 


100 


V-F 


70 


67 


96 


V-G 


110 


66 


60 


V-H 


39 


38 


97 


V-I 


76 


62 


82 


V-J 


2 


0 


0 



*5ourcft: Report of School Visit Fonas Completed During 1972-73* 
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2. TEACHER TRAINING ACTIVITIES 

Another approach to Improving the skills of the classroom teacher » 
teacher training* may be defined as the set of organized and structured 
courses » workshops and other enrichment activities available to Atlanta 
teachers. Participation In one or another of the teacher training activities 
almost always occurs as a result of a decision by the teacher • A teacher may 
enroll In teacher training activities for a variety of reasons: to earn a 
salary increment; to improve general skills; to learn new techniques; to 
obtain an advanced degree; to f^nbance the possibility of promotion within 
the school system's hierarchy. While a principal » a Resource Teacher or 
other personnel may suggest that a teacher enroll in a training program, no 
one may compel a teacher to attend. 

There are four m$jor types of teacher training opportunities offered to 
Atlanta teachers: university courses outside the Atlanta school system; 
programs sponsored by the Instruction Division; workshops conducted by Area 
Resource Teachers; and proficiency modules. Each of these teacher training 
activities will be described below. 

a. UNIVERSITY COURSES OUTSIDE THE 
ATLANTA SCHOOL SYSTEM 

A teacher may enroll in a university course for credit towards a 
salary increment or to obtain an advanced degree. The Atlanta Area Teacher 
Education Service (AATES), founded in 1945, publishes each yc^r a list of 
courses offered at local universities which have been approved for salary 
increment credit by the school system. 

Very little is known about which specific teachers are enrolled in such 
courses. Several principals interviewed indicated that teachers at their 
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schools were enrolled in these courses. Vlhen a teacher successfully com* 
pletes a cottrse, the school system is notified. The Personnel Division 
maintains the payroll records for the school system and is notified by the 
Instruction Division when a teacher has completed sufficient training to be 
entitled to a salary Increase. 

b. INSTRUCTION DIVISION PROGRAMS 

The Instruction Division sponsors a variety of courses , conferences 
and workshops for Atlanta staffs Including teachers. This Division is also 
responsible for the orientation program which each new teacher In Atlanta is 
required to complete. 

Training activities sponsored by the Instruction Division usually have 
salary increment credit associated with them and are known In Atlanta simply 
as "in-service" activities. At the beginning of each school year, a list of 
the courses or workshops to be offered during the year is distributed to 
Atlanta staff. The list of workshops or In-service programs is amended 
throughout the year^ as new courses are offered. 

Final data were not available on participation In 1972-73 programs 
sponsored by the Instruction Division. There was no Indication, however, 
that the pattern of participation would differ substantially from that for 
the preceding school year. According to the school system's data, 57 
approved courses were offered by the Instruction Division In 1971^72. The 
courses varied in subject matter, length and the kind of participants who 
enrolled. Subjects relevant to elementary teachers Included arei art (2 
courses), communications skills (1 course), English (2 courses), mathematics 
(2 courses), science (3 courses) and social studies (7 courses). 
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A total of 1,507 staff members participated in those courites. Atlanta 
records do not indicate how many different teachers were Involved In these 
in-service programs. The school system does not maintain permanent or 
historical records either on which staff members have received In-service 
training in a "^Iven subject or what the training may have Involved. At no 
time are principals officially or systematically informed which teachers are 
enrolled in an In-service program. The teacher usually Informs the principal 
directly, but no mechanism exists to give the principal or Area Superinten- 
dent a record of which teachers are enrolled. 

The Instruction Division is also responsible for organizing an orien- 
tation program for each new Atlanta teacher. The Instruction Division 
establishes guidelines for the program which require the new teacher to 
attend special meetings, to observe an experienced teacher and to visit 
local points of interest. Each Area Office tailors the contents of one- 
half the orientation program to meet the needs of the Area. 

c. AREA WORKSHOPS 

Resource Teachers conduct workshops for teachers and other staff 

of the schools in each Area. During 1972-73, Resource Teachers in four of 

the five Areas provided informal, n about the workshops conducted through 

the Area Office. Information on 204 In-servlce programs was provided by 36 

different Resource Teachers on the Report of Workshop or In-Service Program 
23 

form. No forms were received from Area II Resource Teachers. 

Workshops were offered on a variety of topics: reading, art, team 

teaching, classroom management and testing, to name but a few of the subjects. 

Reading (or language arts) was named in more than 25 percent of the forms as 
23^ See Chapter III, p. 72 for a description of the form. 
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the topic of the workshop. No other topic was so often the subject of an 
Area workshop. Only 6 percent of the workshops dealt with arithmetic, 
although 15 percent of the teachers surveyed In conj unction with this pro*" 
ject Indicated a desire for In-service programs In arithmetic. Approximately 
10 percent of the workshops dealt with classroom organization or management 
or the planning of the instructional program. 

A typical Area workshop is organized and conducted by Area Office staff 
for the benefit of all teachers in the Area. However, more than one- third 
of the workshops were held at individual schools and were attended exclu-- 
sively by the teachers at the school where the workshop was held. In one 
Area, 11 workshops were organized for the faculty at a newly opened school. 

More than half the workshops were attended by more than ten persons; 
attendance ranged from a high of 100 to a low of four persons. The typical 
program was conducted after school hours and lasted about two hours. Nearly 
three-fifths of the workshops were completed in a single session; the 
remainder were continued over several sessions. Some Area workshops spanned 
several calendar months and met on a weekly basis. 

d. PROFICIENCY MODULES 

Proficiency modules represent a new direction in In-servlce training: 
a series of structured individualized instruction for teachers. The modules 
were developed several years ago by the school system and the University of 
Georgia, as a part of the Comprehensive Instructional Program (CIP). To date, 
modules have been completed in reading and arithmetic. A module is designed 
80 that a teacher takes a test to determine proficiency in teaching math or 
reading. The results of the test determine weaknesses of the teacher, who 
then may choose one of several learning paths to gain needed proficiency. 
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Each teacher progresses through the module at his/her own rate, completing 
one topic at a time. The School Board in late December 1972 adopted a 
policy that probationary teachers who do not satisfactorily demonstrate pro- 
ficiency in teaching must complete at least one module before being advanced 
to tenured status. loqplementation of this policy is only just begixinins. 

Successful completion of a set of proficiency modules carries with it 
salary increment credit, and the overall supervision of modules rests with 
the Instruction Division. Day-to-day management of the modules falls to the 
Area Office; Resource Teachers assist classroom teachers with the jT^odules by 
providing needed materials and by being available at specified times for 
assistance. The classroom teacher must pass a written test on the contents 
of each module unit and then successfully demonstrate the newly acquired 
proficiency in a classroom situation. 

Each Area Office was asked to provide a list of participants in the 
modules and to indicate which teachers completed the module program. Area 
III does not use the modules, but the other four Areas provided some Infor- 
mation about participation. In Areas II and IV, a total of 42 teachers 
enrolled in the module progran; these Area Offices did not indicate how many 
teachers completed the modules. In Areas I and V, a total of 66 teachers 
enrolled In the modules, but only 15 completed the module program. 

E. CONCLUSION 

This chapter has focused on three areas of the Atlanta school system 
that affect the educational process as it occurs in the classroom. Changes 
in staffing assignments, in the instructional programs, and in the skills of 
classroom teachers have been discussed in terms of the decisions that are 
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made, the actions that result and the actors who Influence them. 

The process described Is not a systematic one; decisions are made In 
response to a variety of pressures. Actions are often taken on a case-by- 
case or ad hoc basis. Information about the performance of students rarely 
enters the decision process. 

The next chapter describes the effects observed when Information on 
school performance was Introduced during the 1972-73 school year. Partic- 
ular emphasis Is placed on effects of signals on the three activities 
described In this chapter. 
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CHAPTER V 

EFFECTS OF THE INTRODUCTION OF SIGNALS 

This chapter presentis the reactions of Atlanta personnel to the idea 
of comparing relative performance and then discusses the impact of slgn&l 
Information on the three types of activities discussed In the previous chapter 
staffing, the Instructional program and Improving the skills of the staff. 

A. INTRODUCTION 

At the outset of the current study, two different kinds of signal 

24 

materials were distributed to Atlanta staff; (1) booklets containing a 
brief explanation of the derivation of signals, the signals and mean achieve- 
ment scores for each school In an Area; (2) sheets displaying the signals 
for all schools in an Area for each of two years (1971 and 1972) « A smaller 
booklet was prepared for each school principal containing the explanation of 
the derivation of signals and the signals and mean achievement scores for 
his/her school. 

The distribution of these materials began In November 1972. Signal 
booklets were never distributed directly to teachers, since the signals were 
viewed primarily as a management tool for use by school officials above the 
school level. The extent to which signals were made available to teachers 
was left to the discretion of the principal. [Principals of six schools 
requested and received additional booklets for distribution to teachers.] 

24^ Figures II-3 and II-4, p. 31 provide an Illustration of the signals 
for a school. 



106 



Signal materials were distributed to principals and Area Resource Teachers 
through a series of briefings organized by each Area Superintendent. Signal 
laaterlals were explained and distributed to Instruction Division and Person- 
nel Division staff In December 1972 and January 1973. 

The effects of the Introduction of signals in Atlanta will be discussed 
in the next two sections of this chapter. The idea of signaling relative 
performance was generally favorably received. Since the impact of 
signal information is partly a function of how the information is received, 
the next section of this chapter describes the reactions of school system 
officials to the idea of signaling relative performance. The final 
section examines the Impact of signals on the activities described in 
the preceding chapter. 

B. REACTIONS TO SIGNALS AMONG 
SCHOOL OFFICIALS 

The reaction to signals of relative performance throughout all layers 
of the Atlanta administrative structure was generally positive. There was 
widespread praise for the format used to present a massive amount of data 
in a clear, concise manner. Two Area Superintendents noted that previously 
they had never had time to examine student performance as measured by 
achievement test results because of the volume of data Involved. The com- 
puter print-out of achievement test results for the schools in a single 
Ar?a is usually several Inches thick. Principals also commended the format 
and the handy size of the signal booklets. 
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Reactions to the Idee of comparing relative performance of schools were 
somewhat more varied. Atlanta staff at all levels endorsed, In principle, 
the comparisons of schools which have similar student populations. 
Especially appealing to some principals was the Idea of comparing performance 
among Atlanta schools, rather than with the national norm established by the 
achievement test manufacturers. 

The same ambivalence toward the use of achievement test results to Judge 

performance which has been expressed by the larger community of educators was 

echoed by Atlanta officials. While admitting their use of achievement 

results to confirm opinions about teachers, curriculum or students, school 

officials (especially principals) expressed concern over the reliability and 

appropriateness of achievement tests as an evaluative tool. 

The signaling technique relies on participation in the free and reduced- 
25 

price Itmch program as the criterion for identifying similar schools. 
Criticism of the use of subsidized lunch participation in generating signals 
came almost exclusively from a few principals and Resource Teachers. 
They expressed reservations about the use of free and reduced-price 
lunch data to describe the socioeconomic composition of a school. Because 
the income cut-off point for eligibility to participate in the program is so 
low, principals of some schools where small numbers of students receive sub- 
sidized lunches felt that the variable overstated the economic composition 
of the school. That is, two schools — one composed of the children of corpora- 
tion presidents and the other, children of assembly line workers — ^may both 
have comparable rates of participation in the subsidized lunch program, but 

would not be expected to have similar patterns of achievement. 

25. See Chapter II, p. 25 for the discussion of how the variable Is 
used in conjunction with mean achievement to generate signals of relative 
performance. 
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Some recipients of signals suggested that, because of the controversy 
over school desegregation and the unusual degree of student mobility due to 
as yet uncertain busing patterns, the variable was not suitable at the 
present time; these officials usually conceded that subsidized lunch partici- 
pation would be a satisfactory socioeconomic indicator when the situation 
becomes more stable. Suggestions for other variables to use in lieu of or 
in addition to free and reduced-price lunch participation included neigh- 
borhood stability, teacher and/or pupil attendance data, community Involve- 
ment, parental interest in the school, and evaluations of teacher performance 
No one Interviewed could offer a satisfactory source of data presently avail- 
able in Atlanta for measuring these attributes. 

26 

In the initial round of interviews. Area Superintendents, Resource 

Teachers and principals were asked to account for the existence of particular 

27 

red or blue signals. In the second round of interviews, these officials 

were questioned about how signals had been used or who in the scUool system 

could have most use for the information. 

Few principals expressed surprise at the signals for their schools; 

this reaction lends operational support to the statistical evidence of the 

validity of the signals. The presence of a blue signal in a grade was 

generally attributed by the principals to the success of a teacher or 

teachers. The presence of a red signal was generally attributed to student 

discipline problems, student transiency, overcrowding, the absence and/or 

inexperience of particular teachers. Principals had more difficulty 

~ 26^ See Chapter III, p. 68, for a description of these interviews. 

27. The reader is reminded that a red signal (either full or half 
red) indicates relatively low student performance and a blue signal (either 
full or half blue) indicates relatively high performance. Most grades 
receive a neutral signal indicating no extreme of performance. (See Chapter 
II.) 
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accounting for situations In which the signals for the same grade In math 
or reading were different. Many principals wanted to know why certain grades 
were signaled and not others; they felt that explelmlng the reasons for 
signals would lead to the development of solutions to problem situations. 

Principals were very Inconsistent In tUelr responses to questions about 
the usefulness of signal Information. Nearly 80 percent of the principals 
interviewed felt that signals could be of most use to principals and teachers, 
while less than half felt that Area personnel could make best use of signal 
information. When questioned about the routine Interactions between 
principal and teacher, few principals felt signals would have any effect on 
those interactions. The principals did suggest that signals could affect 
thiR relationship between the school and the Area Office. They predicted 
that the signals might, cause both the Area Superintendent and Resource 
Teachers to pay more attention to grades with red or blue signals, that 
Resource Teachers might visit those grades more often, that Area staff might 
set up workshops to deal with problems suggested by patterns in the signals. 
Principals were less certain about the effects of signal information on the 
activities of the Instruction Division or the Personnel Division. 

Resource Teachers and Area Superintendents had more difficulty in 
interpreting signals for specific schools and grades because the impressions 
on which they base their Judgments of performance are not equally current for 
all schools. The presence of a red or blue signal tended to be explained in 
terms of the "quality of teaching" in that grade or the overall "quality" of 
the faculty at the school. Other explanations for signal s-^elther red or 
blue — included curriculum changes, test administration, teacher attitudes, 
and the presence of "unusual" students at the school. 
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Both Area Superintendents and Resource Teachers tended to focus on red 
signals. Several Resource Teachers planned to examine the reading programs 
In grades with red signals In reading or In schools with several red signals. 
One Area Superintendent was so surprised by the red signals at two schools 
In the Area that he changed the assignment of one of the reading Resource 
Teachers to Include those schools. 

In summary t the signals of relative performance distributed for the 
first time In Atlanta In late 1972 were well-received, particularly by 
principals. Area Superintendents and Resource Teachers, The underlying 
principle of comparing performance In similar schools was accepted. The 
method of displaying signal Information won praise for Its simplicity. The 
transition from accepting the Idea In principle to the application of signal 
Information to decisions by school officials was much more difficult to 
make. The remainder of this chapter discusses the effects (or lack of 
effects) of signals on (1) the assignment of teachers and the staffing of 
schools; (2) the nhaplng of the Instructional program; and (3) improving 
the skills of the teaching staff, 

C, IMPACT OF SIGNALS ON TEACHER ASSIGNMENT 
AND SCHOOL STAFFING 

From the beginning of this study, Atlanta officials have said that the 
classroom teacher is a key factor In whether or not students progress. As 
reported above, Area Superintendents, Resource Teachers and principals alike 
attributed the presence of red and blue signals to the quality and attitudes 
of the teaching staff. Yet neither the assignment of individual teachers 
nor the staffing of schools has been substantially affected by the intro- 
duction of information about relative performance. Throughout the year. 
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decisions about individual teacher assignments and the staffing of entire 
schools were influenced by considerations of the racial composition of 
faculties and the declining student enrollment. 

Information on relative performance, as reflected in the signals, was 
expected to be useful in hiring new teachers and in reassigning existing 
staff • When signals were explained and distributed to them, Personnel 
Division staff expressed Interest in using signals to identify character- 
istics of teachers in red and blue signaled grades. They hoped to determine 
if training, years of experience, type of certification or teacher turnover 
appear to affect the level of performance among grades In similar schools. 
Thus, signals could be used as an output measure in a research effort to 
determine characteristics of successful teachers. The results of such an 
effort could be used as a screening device In selecting new teachers or in 
making teacher assignments. 

Although the Personnel Division staff showed an interest in using 
signals to Improve procedures for screening applicants, there is no evidence 
that any one followed through on this interest. The small size of the staff 
of this Division in reality meant that no single staff meinber had the time 
needed to direct or conduct the research needed to answer questions about 
teacher characteristics. Moreover, the value of signal information to 
Personnel staff is reduced because information about which specific grades 
and schools will require new teachers is not available during the normal time 
of recruiting and hiring new teachers. 

The*potential impact of signal information on the reassignment of 
existing staff is higher. As long as signals can be available prior to the 
time most new assignments are made (over the summer and during the first 
weeks of the new school year), then signals can provide information about 
the current and proposed assignment for the teacher in question. 
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Signals played no part In the selection of new teachers, since no new 
elementary teachers have been hired by Atlanta since the Introduction of 
signals. Moreover, according to the Assistant Superintendent for Personnel, 
signals played no part In central decisions about teacher reassignment made 
during the last months of the 1972-1973 school year. Area Superintendents 
provided only one Instance of the use of signal information in reassigning 
staff. One Area Superintendent reasaigned several teachers who had been in 
a school with several red signals to schools with several blue signals in the 
hope that the teachers would be Influenced by the high performance of 
teachers in their new schools. The new assignments were to take effect in 
September 1973. 

D, IMPACT OF SIGNALS ON THE 
INSTRUCTIONAL PROGRAM 

The development of currlculiim guidance for the entire school system 
and the demonstration of new Instructional approaches within the school 
system are usually multi-year activities. Moreover, Title I and other 
federal programs have certain restrictions on where and how money can be 
used. Consequently, it might have been predicted that signal information 
would have little or no impact on decisions relating to the instructional 
program in the short period of this study. At best, it might have been 
predicted that signals would be used by Instruction Division staff to 
investigate the characteristics of the instructional program in grades 
where performance was either relatively high or relatively low. There is 
no evidence to show that signals were even considered in decisions involving 
the shape of the instructional program for the school system. 
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The elementary school curriculum is being revised. Aa the new curriculum 
guidelines are developed, they are being field-tested In a sample of schools. 
The number of schools piloting the new curriculum by September 1973 had 
doubled to 20. There has been no quantitative evaluation of the new curricu- 
lum to date, nor is one planned. Neither performance Information in general, 
nor the signals in particular, seem to have affected the choice of pilot 
schools or the appraisal of the curriculum being developed. 

The Comprehensive Instructional Program (CIP) is another exas;>le of an 
instructional activity which is aimed at Inqproving student performance. 
When the program began in 1970, 50 of Atlanta's elementary schools were 
designated GIF Intensive schools, on the basis of a combination of economic 
need (using 1965 data) and fourth grade reading performance. The evidence 
Is that some of the 50 schools received more attention than the other 75 
Atlanta elimientary schools; some did not. In fact, as was reported in 
Chapter IV, it was left to the discretion of each Area Superintendent to 
decide how CIP Resource Teachers were to be used. The CIP conducted its own 
evaluation for the first two years of its existence, but this effort has been 
abandoned. Signals might be used as an indicator of where Intensive assist- 
ance should be provided, and changes in signals could serve as an indicator 
of program effectiveness. While the Instruction Division staff of the CIP 
were familiar with signals, they gave no indication of having used signals 
in the operation of the program. 

Signals did not stimulate Investigations of the characteristics of the 
instructional programs in red or blue signaled grades. For exanqple. In- 
struction Division staff could cite no attempts to develop empirical evidence 
of the success or failure of particular texts. Although several principals 
reported adopting new textbooks for the coming year signals played no part 
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In the decision about which texts to adopt. Nor did Resource Teachers re- 
port using signals in decisions about particular texts or materials. While 
30 percent of the first grades in one Area received red signals in math and 
40 percent of the third grades in another Area received blue signals in math, 
neither Area Office personnel nor curriculum development staff reported 
having given any thought to what might have occurred in those grades. Neither 
Area Superintendents nor principals reported using signals in discussions 
about the instructional program for the schools. One principal in the survey 
did use the signals to support a decision to restructure the arithmetic pro- 
gram in a grade which had received red signals for two years in a row. 

E. IMPACT OF SIGNALS ON EFFORTS TO 
IMPROVE THE SKILLS OF THE 
TEACHING STAFF 

Two mechanisms for improving teacher skills— direct assistance to 
classroom teachers and structured teacher training activities — were described 
in the preceding chapter. The remainder of this chapter is concerned with 
whether signals had an Impact on the provision of direct classroom assistance 
to teachers. The evidence of that is largely quantitative; in the case of 
In-servlce training, the evidence is mostly qualitative and based on personal 
Interviews. 

No Atlanta official reported using signal Information in the provision 
of teacher training. The Resource Teachers who organized and conducted 
workshops could cite no Instances of the use of signals in decisions about 
the subjects to be offered or the location of the workshops. Neither 
principals nor Resource Teachers remembered directing a teacher from a red 
signaled grade to enroll in an in-service program or proficiency module, 
although both principals and Resource Teachers did make such recommendation 
without regard to signals. 
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Answers to questions about the Impact of signals on Resource Teacher 
assistance to classroom teachers are more complex. As described In Chapter 
Illy Resource Teachers provided Information on visits to schools during the 
1972-1973 school year on the Report of School Visit form. The recorded In- 
formation was processed to create a data base for analyses of Resource 
Teacher assistance during the 1972-1973 school year, to determine If signals 
had a measurable effect on the distribution of that activity among elementary 
grades. Information was not available on the pattern of Resource Teacher 
activity prior to September 1972, nor was there any agreement a priori as 
to what should be the pattern of direct assistance by Resource Teachers. 
Consequently, the analyses have been restricted to variations within the 
pattern of activity for the 1972-73 school year. In particular, the follow- 
ing question was addressed. 

• Can variation In patterns of Resource Teacher activity during the 
1972-^1973 school year be associated or contrasted with variation 
In patterns of signals for 1972? 

That Is, It was assumed that the signals based on 1972 achievement, which 

were distributed In late 1972, could affect the pattern of Resource Teacher 

Activity for the 1972-73 school year. 

By using different measures of activity and different definitions of 

patterns of activity, a variety of "associations'* or "contrasts" can be 

examined to provide evidence of the effect or lack of effect of signals. 

Three general measures have been used In the analysis to categorize Resource 

Teacher assistance to teachers: grades visited^ time expended on visits, and 

the frequency of visits. As will be seen shortly, a variety of specific 

measures can be defined for each of the three. Two patterns of Resource 

Teacher activity have been considered— the distribution of Resource Teacher 
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activity over all the grades that could have been visited and the distribu- 
tion of Resource Teacher activity among the grades actually visited. The 
first distribution was used to answer the question: 

o Among the grades that existed (and conceivably could have been 
vxsited) did Resource Teachers give a disproportionate amount 
of assistance to grades having a particular type of signal? 

The second distribution was used to answer the question: 

• Among the grades that were in fact reported as having been 
visited, did Resource Teachers give a disproportionate amount 
of assistance to grades having a particular tjnpe of signal? 

Both questions were addressed in order to take into considerattion restrictions 
on Resource Teacher activity which may have .prevented them from providing as-« 
sistance to some grades, independent of the type of signal that existed. 

All of the analyses presented below are based on those visits by Re- 
source Teachers which involved at least one classroom teacher in Grades 

28 

1-7 in primary or elementary schools. The focus on Grades 1-7 was dictated 
by the fact that these are the only grades for which signals existed. Thus, 
instead of the 3,181 Resource Teacher visit forms which were collected and 
used to describe Resource Teacher activities in Chapter IV, the analyses 
of the effects of signals on Resource Teacher assistance to the classroom 
are based on the 1,875 visits involving classroom teachers. The remaining 
1,306 visits were to principals and other school staff not directly 
assigned to the classroom. 

28^ The number of visits and grades used in this chapter are sometimes^ 
different than those used in Chapters II and IV because in those chapters 
different sets of schools were used as bases for analysis and discussion. 
Gere visits to schools (or grades) for which there were no signals (blue, 
neutral or red) in 1972 were omitted. 
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The remainder of this section Is divided Into two parts. The first 
part examines the impact of signals on the distribution of Resource Teacher 
assistance among all grades In Atlanta; the second part examines the Impact 
of signals on the actual assistance provided. 

1. THE IMPACT OF SIGNAL IN170RMATI0N ON THE 
ALLOCATION OF RESOPRCE mCHER ASSISTANCE 

The results presented below compare the observed distribution of Re- 
source Teacher assistance to classroom teachers with vhat would have been 
expected if assistance had been given in proportion to the total number of 
all grades with each type of Blffo&l. If the observed and expected distri- 
butions do not differ significantly, then it can be concltided that the Re- 
source Teachers did not give a disproportionate amount of assistance to 
grades having a particular type of signal. Such a result could be inter- 
preted as an indication that signal information did not affect the allocation 
of Resource Teacher assistance to classroom teachers. This interpretation 
seems reasonable, since the aim of the signals Is to identify extremes of 
performance on the assumption that the system will react to this information 
and provide evidence of disproportionate attention to the grades exhibiting 
extremes of performance. 

Two terms are used throughout the analyses presented below: the expected 
distribution and the observed distribution of Resource Teacher assistance. 

29. The distribution of Resource Teacher assistance in prior years may 
have been quite different from that observed in 1972-1973. Thus, it could 
be argued that the introduction of signals vras associated with a redistri- 
bution of Resource Teacher effort. Baseline data on the distribution of as- 
sistance prior to the introduction of signals are not available, rendering 
in^ossible such pre/post con^arisons. 
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Expected distribution ! The distribution which vould be expected 
If Resource Teacher assistance were provided In the same propor- 
tion as blue» neutral and red s ignals^O for April 1972 were 
distributed among the grades that existed. 

Observed distribution ; The actual distribution of any measure 
of Resource Teacher assistance provided In 1972-73 to grades 
Identified as having received a blue, neutral or red signal In 
April 1972. 

Analyses have been conducted using the observed and expected dlstrlbu** 
tlons defined In terms of several measures of Resource Teacher assistance: 
grades visited, time expended and frequency of visits. For each measure, 
the observed distribution of Resource Teacher assistance has been calculated 
for all grades in Atlanta and for all grades in each Area. The analyses 
of the distribution of Resource Teacher assistance by Area seemed necessary 
since Resource Teachers are assigned to an Area Office and the pattern of 
their activity might be expected to vary among the Areas. All analyses 
were conducted using the observed and expected distributions defined rela- 
tive to reading and arithmetic signals. The results were similar. There- 
fore, data presented below refer primarily to reading signals. In those 
few cases where different results occur for arithmetic, the exceptions are 
noted. 



a. EFFECT OF SIGNAL INFORMATION ON 

GRADES VISITED BY RESOURCE TEACHERS 

The most basic measure of Resource Teacher assistance is "which 

grades are visited," without regard to the amount of assistance given. 

Resource Teachers state that it is not possible to visit every grade. 

30l Throughout these analyses » the term "blue sisn^l" refers to 
relatively high performance as indicated by a full or half blue signal; 
"red signal" refers to relatively low performance as Indicated by a full or 
half red signal; "neutral signal" refers to performance which is not sig- 
naled as extreme. (See discussion of signals In Chapter II.) 
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A total of 806 grades uhlch existed In 1972"*73 tacelved a signal In 
April 1972: 10.8 percent received blue signals; 81.6 percent received 
neutral signals; 7.6 percent received red signals In reading. During 1972- 
1973, Resource Teachers reported at least une visit to 485 of those grades 
(60 percent). .The observed distribution of grades visited, according to 
type of reading signal was 10.3 percent with blue signals, 83.1 percent 
with neutral signals and 6.6 percent with red signals. Thus, the observed 
distribution of grades visited is nearly identical to the existing or 
expected distribution of signals. The observed and expected distribution 
do not differ by more than 1.5 percent. 

To determine if the differences were statistically significant, the 
two distributions were compared using a chl--square (or X^) test with two 
degrees of freedom and a sample size determined by the number of visits 
that actxially occurred. The results indicate that the detected difference 
would be statistically significant only at a confidence level on the order 
of 20 percent or more. That is, the difference is not significant at any 
of the typically used levels of confidence, aiich as 1 percent, 5 percent 
or 10 percent. In operational terms, a 1.5 percent difference means that 
only 12 grades out of the 806 total (or 10 out of the 658 neutral grades) 
were visited that might not have been expected to be visited if assistance 
were provided proportionally. Similar results occur when the analyses are 
conducted on an Area basis and when arithmetic signals are used. The data 
used to obtain these results are presented in Table V*-l. 
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TABLE V-1 

EXPECTED AND OBSERVED DISTRIBUTIONS OF RESOURCE TEACHER VISITS, 
BY AREA AND BT SIGNAL, BEADING 1972-73 



GRADES 
CONSIDERED 


EXPECTED ^ 
DISTRIBUTIONS^ 

TEl- m Of SIQCALS 
Blue Neutral Red 


OBSERVED ^ 
DISTRIBUTIONS^ 

rscBVT or sioiau 

Blue Neutrel Red 


DIFFERENCE 
SIGNIFICANT AT 
LEVEL OF: ^ 
15? 5X 


ALL GRADES 


10.8 


61.6 


7.6 


10.3 


83.1 


6.6 


NO 


NO 


AREA I 


15.8 


82.5 


1.7 


13.0 


84.8 


2.2 


NO 


NO 


AREA II 


3.2 


86.4 


10. A 


3.2 


•7.3 


9.5 


NO 


NO 


AREA III 


21.6 


69.0 


• 9.4 


21.^; 


70.9 


7.8 


NO 


NO 


AREA IV 


3.3 


. 87.8 




3.1 


87.5 


9.4 


NO 


NO 


AREA V 


U.l 


82.8 


6.1 


lO.l 


85v9 


4.0 


m 


NO 



1. Dletrlbutlon thet vould be earyected If vielte vere proper tionel to the dletrlbutlon of elgnele* 

2. Obeerved dletrlbutlon of Reeource Teecher vlelte hy type of elgnel. 

3. Stetletlcel elgnlf Icence deteralaed bf • teet with t«o defreee of f reedoa end e eeaple elze equel 

to the nu^r of vlelte thet occurred. 



In summary, when the number of grades visited is used as a measure of 
Resource Teacher assistance, there is no evidence to indicate that a dis- 
proportionate amount of assistance was given grades having a blue or red 
signal in reading or arithmetic. The same result is obtained when considering 
Resource Teacher visits to all schools in the system or the schools In each 

of the five Areas. 

However, as has been pointed out, a measure based on whether or not a 

grade was visited at least once during the school year does not take into 
consideration the amount of assistance given. The number of grades visited 
is simply a measure of the minimum level of effort provided. In order to 
examine the amount of Resource Teacher assistance provided classroom teachers, 
analyses based on time expended on Resource Teacher visits have been per- 
formed. The results of those analyses follow. 
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b. EFFECT OF SIGNAL INFORMATION ON TIME 
EXPENDED DURING RESOURCE TEACHER VISITS 

The length of each visit reported by Resource Teachers provides a means 
of weighting the amount of assistance given in a visit. If each grade had 
only one teacher and each Resource Teacher visited only one teacher at a time, 
then the reported time could be used as a measure of activity without any am*- 
blguity. However, in most Atlanta schools, several teachers are assigned to 
each grade level nnd, on many occasions, Resource Teachers simultaneously 
visit several teachers from one or several grades. Personnel other than 
classroom teachers also participate in many visits. In order to examine the 
impact of signal information as measured by the length of Resource Teacher 
visits, a method had to be found to allocate the benefit, as measured by 
time spent on a visit, to the participants and the grades they represent. 

Since no accepted theory exists to scale the benefit of group versus 
individual meetings between teacher and Resource Teacher, two definitions 
of time were considered: 

(1) Resource Teacher Time; Each grade is allocated a proportion of 
the total time expended by the Resource Teacher equal to the 
proportion of participants who are teachers in that grade. That 
is, if a Resource Teacher spends a total of t^ minutes simul- 
taneously visiting n persons, of whom m are teachers in grade jg^, 
then a measure of time expended on grade ^ during the visit v is: 

T(v,g) - ^. 

(2) Teacher Time: Each grade is allocated a multiple of the total 
time expended by the Resource Teacher equal to the number of 
participants who are teachers in that grade. That Is, If a 
Resource Teacher spends a total of t^ minutes simultaneously 
visiting n persons, of which m are teachers In grade then a 
measure of time expended on grade ^ during the visit v Is: 

T'(v,g) - mt. 

Resource Teacher time assumes that the benefit to each participant in 
a visit (as measured by time) is equally divided among the participants so 
that the sum of the benefit (time) equals exactly the time the Resource Teacher 
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expended. On the other hand, the second measure, teacher time, assumes that 
all participants derive the same amotrnt of benefit as if only one had been 
present. That is, in the second case, benefit to each participant is assumed 
to be independent of the number of teachers present. 

While both measures were used in the analyses performed to determine if 
signal information had an effect on Resource Teacher assistance, the results 
obtained using Resource Teacher time did not differ from the results obtained 
using teacher time. Thus, while data which appear in the succeeding para- 
graphs refer only to Resource Teacher time, the findings are generally appli- 
cable to teacher time. 

31 

Resource Teachers reported spending 1,978 hours assisting classroom 
teachers of Grades 1-7. If these hours had been expended in direct propor- 
tion to the signals assigned to the grades in reading, then 10.8 percent of 
the time would be expected to be spent in blue signaled grades, 81 ..6 percent- 
in neutral grades and 7.6 percent in red signaled grades. In fact, the ob- 
served distribution of Resource Teacher time was as follows: 9.6 percent in 
blue signaled grades, 82.5 percent in neutral grades, and 7.9 percent in red 
signaled grades. As with the distributibh of grades visited, tbe^^^o^ 
distribution and the expected distribution of Resource Teacher time were very 
similar; a comparison of the difference is not significant at a confidence 
level of 25 percent or less. 

The differences between the observed and the expected distribution of 
Resource Teacher time are even less significant when viewed In light of the . 
operation of the school system. For example, a difference of 1 percent be- 
tween observed and expected amounts of Resource Teacher time represents less 

31, The total time reported by Resource Teachers, including assistance 
to principals and other staff not directly assigned to Grades 1-7 was sub- 
stantially greater than 1,978 hours. 
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than 20 hours out of the total amount reported over the entire school year« 
Thus, in terms of Resource Teacher time, the sum of the absolute difference 
between observed and expected effort given to blue, neutral and red signals 
represents less than 47 hours of reported Resource Teachers' time over the 
whole school year — hardly an operationally significant difference for the 
combined efforts of the 52 Resource Teachers who submitted reports. The 
differences detected for Resource Teacher effort relative to arithmetic 
signals are even smaller than for reading. 

Different results are obtained when the distributions are calculated 
on an Area basis. Tn particular, the difference between the observed 
distributions of Resource Teacher time and the expected distribution of Re- 
source Teacher time relative to reading signals is statistically significant 
at the 1 percent confidence level for Areas I, III and IV, When arithmetic 
signals are used, the detected differences are statistically significant at 
the 1 percent level only in Area III. When the level of statistical con- 
fidence is 5 percent, then detected differences relative to arithmetic signals 
become significant for Area I. These results are presented in Table V-2 below, 

TABLE V-2 

EXPECTED AND OBSERVED DISTRIBUTION OF RESOURCE TEACHER TIME, 
BY AREA AND SIGNAL, 1972-73 



a. Rttsdlns titulo 



GRADES 


EXPECTED , 
DISTRIBUTIONSi. 


OBSERVED . 
DISTRIBUTIONS^ 


DIFFERENCE 
SIGNIFICANT AT 


CONSIDERED 


PERCEirr OF SIGNALS 


PERCENT OF SIGNALS 


LEVEL Oft^ 




Blue 


19eutral 




Blue 


KeutTftl 


Red 


1% 


sz 


ALL CKADES 


10.8 


81.6 


7.6 


9.6 


82.5 


7.9 


HO 


KO 


AREA I 


15.8 


82.5 


1.7 


9.9 


87.2 


2.9 


YES 


YES 


Am II 


3.2 


86.4 


10.4 


1.9 


89.0 


9.0 


NO 


no 


AKEA III 


21.6 


69.0 


9.4 


35.6 


62.0 


2.4 


YES 


YES 


AUA IV 


3.3 


87.8 


8.8 


2.3 


84.7 


13.0 


YES 


YES 


Am V 


11.1 


82.8 


6.1 


9.9 


•1.9 


3.1 


NO 


NO 



(Continued on following page) 
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TABLE V-2 (continued) 

b. Arithmetic ll«Mla 



GRADES 


exfected , 
distributions! 


OBSERVED « 
DISTRIBUTIONSi 


DIFFERENCE 
SIGNIFICANT AT 


CONSIDERED 


FEICEirr OF 8IGMALS 


PERCENT OF SIGNALS 


LEVEL OF:^ 




Slut 


VMtnl 


R«d 


Blua 


Ntutrtl 


Ud 


IX 


$x 


AIX CXADES 


8.4 


S5.9 


5.7 


7.6 


86.6 


5.7 


WO 


NO 


AREA I 


10.0 


89.2 


0.8 


6.1 


92.6 


1.4 


NO 


Y2S 


AREA II 


2.6 


89.6 


7.8 


1.8 


90.8 


7.4 


NO 


NO 


AREA III 


20.5 


74.3 


5.3 


32.5 


61.2 


6.3 


YES 


YES 


AREA IV 


3.3 


87.8 


8.8 


2.3 


90.3 


7.4 


NO 


NO 


AREA V 


6.1 


89.4 


4.4 


4.1 


91.3 


4.6 


m 


m 



1. Dl«l:rlbtttlM that vtuU to MHCtcd If vlaltt ««r« no^ortlMUl to tYm ^iatrlbutlOK mi tltnAU. 

2. ObMnrtd dUtrlbutloo of lofoureo Toacbtr vloltt bjr tff of oltnal. 

3. ScotlotlcAl •Ignlf iMBco dotonlMd « tttt vlth tvo dtgtMft of f roodo* nd • oMiplo olio •q;Ml 
to th« Buabor of vUlto that occur rod. 



While the differences between the observed and the expected distribution 
of Resource Teacher time In Areas I, III and IV were statistically significant 
at the 1 percent level In reading, the differences appear to have been in- 
significant in terms of the operation of the school system. Moreover, there 
was no common pattern across Areas. In Axea I, there was a slight shift In 
attention (as measured by Resource Teacher time) away from blue signaled 
grades towards grades vlth a neutral signal. In terms of actual Resource 
Teacher time, the maximum detected difference In Area I represented only 22 
hours of Resource Teacher effort out of a total of 368 hours reported. In 
Area III, Resource Teachers spent more time than expected with blue signaled 
grades (In reading and arithmetic) at the expense of both red and neutral 
signaled grades. The differences between observed and expected values In 
the blue signaled grades of Area III represented about 31 hours out of the 
290 hours reported by Resource Teachers In that Area. In Area IV, effort 
was shifted from neutral signaled grades to red signaled grades. The 

erJc 



differential effort on red signaled grades represented 37 hours out of the 
890 hours reported. 

Thus, except possibly in Area III, data collected on how Resource 
Teachers expended tine provide no evidence to Indicate that grades were 
singled out for attention according to the signal they received during the 
previous school year. This finding Is particularly true when the distribution 
of time Is considered relative to math signals. 

In still another effort to examine the distribution of direct assistance 
provided by Resource Teachers, a measure based on the frequency of visits was 
utilized to weight the attention given to grades visited. These analyses are 
discussed below. 

c. EFFECT OF SIGNAL INFORMATION ON THE 
FREQUENCY OF RESOURCE TEACHER VISITS 

As with the time expended, a measure of Resource Teacher activity based 
on frequency of visits provides a means of weighting the amount of attention 
given to grades having different types of signals. If each grade had only 
one teacher and a Resource Teacher visited only one teacher at a time » then 
the frequency of visits to a particular grade would be equal to the number 
of visits reported. However, as has already been pointed out, most grades 
have several teachers and many visits Involve several teachers from one or 
more grades. Therefore, as with the "time" measures. It was necessary to 
adopt sone mechanism to distribute each visit among all the participants. 

Once again, two measures for the frequency of visits to classroom 
teachers were defined. They parallel the measures defined for time spent 
assisting teachers. One measure allocates each Resource Teacher visit among 
all participants In the visit, on the assuoqptlon that the benefit to the 
^'Ttlclpants Is shared equally. The other measure assumes that the benefits 
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to each participant is independent of the number of participants and, there- 
fore, allocates one visit to each participant. 

When both measures were used to analyze Resource Teacher data, the 
results were the same regardless of how frequency of visit was defined. 
The results are also nearly Identical with those reported above when time 
was used as a measure of Resource Teacher activity. In particular, when 
all grades are considered without regard to Area, the observed distribution 
of visits is quite close to the distribution which would have been expected 
if the visits had been allocated in direct proportion to the number of 
grades having blue, neutral or red signals. For example^ for grades 
signaled in reading, the observed distribution of visits was 10.4 percent 
to blue signaled grades, 81.6 percent to neutral signaled grades and 8.1 
percent to red signaled grades, almost identical to the expected distribu- 
tion of 10.8 percent blue, 81.6 percent neutral, and 7.6 percent red. Using 
2 

a X test of significance, the differences between these distributions are 
not significant at confidence levels up to 50 percent. The maximum dif- 
ference between the distributions occurs relative to the red signs^ls and 
that is only a difference of O^i 5 percent. This represettts a total of less 
than 10 of the 1,875 visits reported by all Resource Teachers for the 
entire year. The results of the analyses of Resource Teacher frequency 
of visits may be found in Table V-3. 
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TABLE V-3 

EXPECTED AND OBSERVED DISTRIBUTION OF THE FREQUENCY OF 
RESOURCE TEACHER VISITS BY AREA AND BY SIGNAL, 1972-73 



a. Raadlnt Signal* 



GRADES 
CONSIDERED 


expected , 
distributions! . 

PERCENT OF SIGNALS 
Blue Keucral Red 


OBSERVED , 
DISTRIBUTIONS^- 

PERCENT OF SIGITALS 
Blue Keutrsl Red 


DIFFERENCE 
SIGNIFICANT AT 
LEVEL OF: ^ 
11 5% 


ALL GRADES 


10.8 


81.6 


7.6 


10.4 


81.6 


•8.1 


NO 


NO 


AREA I 


15.8 


82.5 


1.7 


9.8 


87.2 


3.0 


TES 


YBS 


AREA II 


3.2 


86.4 


10.4 


2.1 


85.5 


11.4 


NO 


NO* 


AREA III 


21.6 


69.0 


9.4 


36.9 


59.5 


3.6 


YES 


YES 


AREA IV. 


3.3 


87.8 


8.8 


2.2 


85. i 


12.5 


YES 


YES 


AREA V 


11.1 


82.8 


6.1 


9.3 


87.0 


3.7 


NO 


NO 


b. Arithmetic Signals 


ALL GRADES 


8.4 


85. 9 


5.7 


8.5 


86.2 


5.3 


■ NO 


NO 


AREA I 


10.0 


89.2 


0.8 


5.8 


92.8 


1.5 


NO 


YES 


AREA II 


2.6 


89.6 


7.6 


2.2 


89.8 


8.0 


NO 


NO 


AREA III 


20.5 


74.3 


5.3 


34.1 


60.7 


5.1 


YES 


YES 


AREA IV 


3.3 


87.8 


8.8 


2.3 


91.2 


6.5 


NO 


YBS 


AREA V 


6.1 


89.4 


4a4 


3.4 


90.8 


5,7 


NO 


HO 



1: Distribution that would be expected if visita were proportional to the diatribution of aignala. 

2. Observed distribution of Resource Teacher vlaits hj type of aignal. 

3. Statistical significance deterained by a tejit with tvo degreea of freedom and a aaaple aite e^al 

to the number of visits thst occurred. 
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As when time was used as a measure of Resource Teacher assistance, sta- 
tistically significant differences between the observed and expected frequency 
distributions appear In Areas I, III and IV In reading (at the 1 percent level) 
and In arithmetic In Area III (at the 1 percent level) and In Areas I and IV 
(at the 5 percent level). However, only In Area III do the differences ap- 
pear large enough to have some operational significance. 

There Is no pattern to the differences observed among Areas for 
reading. In Area I, red and neutral signaled grades were visited slightly 
more often than blue signaled grades, but In operational terms » that dif- 
ference represented only about 21 visits out of 357 reported. In Area III, 
blue signaled grades were visited more often, at the expense of neutral and 
red signaled grades; the difference amounted to 57 of the 373 visits reported. 
In Area IV, red signaled grades were visited at the expense of neutral and 
blue signaled grades, but the difference represented only 29 visits out of 
793 reported. 

In summary, analyses using the distribution of visits as a measure of 
activity corroborate the results obtained when time was used as a measure of 
activity. That Is, when all grades are considered, reported Resource Teacher 
activity was distributed among grades having blue, neutral and red signals 
In nearly the same proportions as the signals were distributed. And, while 
statistically significant differences between the observed and proportionate 
distribution were detected In Areas I, III and IV, only in Area III were the 
differences large enough to have operational significance for the school 
system. 
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d. FINDINGS 

The preceding analyses have been addressed to the question of whether 
or not Resource Teachers gave a disproportionate amount of assistance to 
grades In 1972-73 which had received a particular type c£ signal In April 
1972. Resource Teacher assistance was measured In terms of the number of 
grades visited, the amount of time spent In assisting teachers In those grades, 
and the frequency of visits to those teachers. It was assumed that If 
signal Information had no effect on the allocation of Resource Teacher as- 
sistance, then the observed distribution of any measure would be proportional, 
to the distribution of blue, neutral and red signals among existing grades. 
The results may be summarized as follows: 

• When all grades were considered without tegard to Area, 
no statistically significant differences were detected 
between the observed distribution of Resource Teacher 
assistance and the expected distribution. This result 
applies when Resource Teacher assistance is measured 

by grades visited, time expended or frequency of visits. 
The result holds when either reading or arithmetic 
signals are used to define Resource Teacher assistance. 

• When Resource Teacher assistance was examined on an 
Area basis, no statistically significant differences 
were detected between the observed distribution of 
grades visited and the expected distribution. The 
result holds when either reading or arithmetic signals 
are used to define Resource Teacher assistance. 

• When either time expended or frequency of visits was 
used as a measure of Resource Teacher assistance, 
statistically significant differences were detected 
in Areas I, III and IV. The greatest differences 
occurred relative to reading signals. Only in 
Area III, however, did the differences appear large 
enough to have any significance in terms of the opera- 
tion of the Area. 



ERLC 



130 



2. THE IMPACT OF SIGNAL INFORMATION ON 

THE RESOURCE TEACHER ASSISTANCE PROVIDED 

The analyses presented In the preceding section focused on the allocation 
of Resource Teacher assistance. This section discusses the variation In Re- 
source Teacher assistance to those grades which were visited at all. The 

grades visited by Resource Teachers were categorized by the type of signal 

32 

each grade received In reading In 1972. The amount of Resource Teacher 
assistance (as measured by time expended and by frequency of visits) to blue, 
neutral and red signaled grades was calculated. The distribution of Resource 
Teacher time and the frequency of Resource Teacher visits for each type of 
signal have been compared to determine whether Resource Teachers gave a dis- 
proportionate amount of assistance to either blue, neutral or red signaled 
grades. There was no way to determine in advance of the study whether 
there should be a difference in the amount of assistance provided or how 
much that difference should be. 

a. OBSERVED EFFECT OF SIGNALS IN TERMS 
OF TIME EXPENDED 

33 

Once again, Resource Teacher time was used as a measure of Resource 
Teacher assistance. The total amount of Resource Teacher time spent in a grade 
over the entire year was determined, and cumulative frequency distributions, 
using intervals of 15 minutes in length, were prepared for blue, neutral and 
red signaled grades. The three cumulative frequency distributions are pre- 
sented In Figure V-1. Each curve in Figure V-1 is presented as a smooth ap- 
proximation to the step function actually calculated. 

32I Each of the analyses discussed in this section was conducted for 
arithinetic, as well as reading. The results were essentially the same for 
reading and arithmetic, and only the results for reading are presented. 

33. See p. 120 for the definition of Resource Teacher time. 
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FIGURE V-1 

CUMULATIVE FREQUENCY DISTRIBUTION OF GRADES 
ASSISTED BY RESOURCE TEACHERS: TOTAL HOURS 
OF ASSISTANCE PROVIDED DURING 1972-1973 




HOURS OF ASSISTANCE PROVIDED 
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As can be seen from observing the three curves In Figure V-1, they are 
remarkably similar. In fact, using the Kolmogorov-Smlrnov Two-*sample Test for 
cumulative distributions, the detected differences for reading are not sig- 
nificant at the five percent confidence level when either the distributions 
are considered for either: 

blue and neutral signaled grades, or 
blue and red signaled grades, or 
red and neutral signaled grades. 

The curves in Figure V-1 differ noticeably only when we consider grades 
in which a large amount of time was expended over the year by Resource 
Teachers. Specifically, approximately 20 percent of the red signaled grades 
visited received at least eight hours of attention but only 6 percent of blue 
signaled grades received that much assistance from Resource Teachers. These 
observations Indicate that Resource Teachers favor red signaled grades when 
expending a relatively large amount of time.^^ However^ the differences are 
neither statistically significant nor large enough to be operationally 
significant. 

The variation among the curves for large time values does have an 
effect on the measures of central tendency (which give weight to extreme 
values of a distribution). In particular, the mean or average time expended 
over the year In grades with each type of signal is as follows: blue sig- 
naled grades — 3.44 hours; neutral grades — 3.74 hours; and red signaled 
grades — 4.24 hours. However, using a t-test, the differences in mean time 
between red signaled grades and blue or neutral signaled grades are not 

34. Note that the expended time is the total amount of time reported 
as being expended over the entire year by all the Resource Teachers. 
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even statlutlcally slgnlflccmt at a 10 percent confidence level. The 
observed differences result from a few red signaled grades which received a 
large amount of Resource Teachei^ when the median Is used as a measure of 
measure of central tendency, the differences are even less. In particular, 
the median amount of time expended in each category of signals is as follows: 
blue — 2.88 hours; neutral — 2.50 hours; and red — 2.92 houra. Clearly, these 
are not operationally significant differences. 

In nummary, while differences can be detected in the distribution of 
Resource Teacher time spent in grades having a blue, neutral or red signal 
in reading, the differences generally are not significant and ^4 not indicate 
a pervasive tendency to provide assistance to grades having a particular 
type of signal. 

b. OBSERVED EFFECT OF SIGNALS IN 
TERMS OF FREQUENCY OF VISITS 

The frequency of Resource Teacher visits was also used as a mea- 
sure of Resource Teacher assistance. The total number of Resource Teacher 
visits to each grade was calculated. Cumulative frequency distributions 
based on the total number of visits reported over the entire year were pre- 
pared for the blue, neutral and red signaled grades. The three cumulative 
frequency distributions are presented in Figure V-2. Once again, the curves 
appear as smooth approximations of the step functions actually calculated. 

As can be observed, the three distributions tend to behave similarly. 
In fact, using the Kalmogorov-Smlmov Two-sample Test, the distributions of 
the frequency of visit measure are not statistically significant when con- 
sidering the distributions for reading relative to; 

35. See p. 124 for the definition of frequency of visit. 
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FIGURE V-2 

CUMDIATIVE mODENCY DIST&XBUTION OF GRADES ASSISTED BY 
R£S(mtCE TEACHERS: TOTAL NUMBER OF VISITS OCCURRING DURING 1972-73 




.5 I 2 7 ■ ■ 12 ■ • • >I6 



NUMBER OF VISITS 

NOTE; THE HORIZONTAL AXIS— THE NIMBER OF VISITS«-HAS BEEN ELONGATED BETWEEN 
0 AND 2 TO DISPLAY THE CUMULATIVE EFFECT OF VISITS THAT INVOLVE SEVERAL CLASSROOM 
TEACHERS. THAT IS» EACH GRADE RECEIVES A PROPORTIONAL CREDIT FOR THE VISIT EQUAL 
TO THE PROPORTION OF TEACHERS FROM THAT GRADE WHO WERE INVOLVED IN THE VISIT. 
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blue and neutral signaled grades, 
blue and red signaled grades, and 
red and neutral signaled grades* 

As with the distributions of time expended, the curve for the dis- 
tribution of visits for grades which had a red reading signal dominates the 
other two curves for small values of the measure; the reverse Is true for 
grades which received a relatively large number of Resource Teacher visits. 
However, the difference between the curves for large values of the "fre- 
quency of visit measure" Is not as great as when "time" was used as a 
measure of activity. Thus, the means or averages for the distributions In 
Figure V-2 do not differ from one another as much as when time was the 
measure of activity. In particular, the mean or average number of visits 
that occurred to grades having blue signals was 1.9 visits; for neutral 
signals, .2.0 visits; and for red signals, 2.6 visits. The medians for the 
three distributions are 2.7, 2.3 and 2.8 visits respectively. Clearly, the 
differences that exist between these measures of central tendency of the 
distributions of Resource Teacher effort among grades having blue, neutral 
or red signals does not have much significance in terms of the operation of 
the school system. 

3, SUMHARY OF FINDINGS ABOUT THE IMPACT OF SIGNALS ON RESOURCE TEACHERS 

The analyses of the characteristics of the Resource Teacher visits 
which were reported in 1972-73 failed to show any operationally Important 
tendency for Resource Teachers to single out either blue, neutral or red 
signaled grades for special attention. Analyses have also been conducted 
on a grade level basis as well as on an Area basis. In a few cases, sta- 
tistically significant differences were detected between the observed and 
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expected distributions, but none of the differences have obvious operational 
significance nor are they as great as those detected in the Area analyses. 
Furthermore, there is no apparent pattern of differential assistance tc a 
particular grade or group of grades having a particular type of signal. 
For these reasons , the detailed discussions of the grade-by-grade analysis 
have been omitted from this report. 

The results and Implications of the findings on the Impact of signals 
that have been reported in this chapter are discussed in Chapter I of 
this report. 



